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Much  has  been  written  about  the  need  for  software 
safety;  however,  very  little  of  the  literature  addresses  the 
practical  application  of  a  software  safety  program.  This 
report  examines  the  software  safety  framework  for  DOD  weapon 
system  development  and  assesses  the  current  state  of 
software  safety  programs.  The  report  is  designed  for  system 
safety  engineers  and  managers,  program  managers,  as  well  as 
system  and  software  engineers.  While  a  background  In  DOD 
weapon  system  acquisition  and  software  development  will  help 
the  reader  understand  the  terminology,  Appendix  A  provides  a 
glossary  of  terms. 
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Abstract 

This  research  analyzed  the  relationships  between  the 
DOD  software  development  process,  system  safety 
requirements,  and  current  structured  software  safety 
analysis  techniques.  The  current  state  of  software  safety 
was  assessed  within  the  aerospace  industry  and  DOD,  and  a 
training  program  for  DOD  System  Safety  Managers  was 
developed. 

A  telephone  survey  was  conducted  to  gather  information 
on  current  software  safety  analysis  techniques  and 
methodologies.  Personal  interviews  were  conducted  with 
Aeronautical  System  Center  System  Safety  Managers  to  gather 
data  on  job  perception  and  perceived  training  needs. 

The  results  of  the  study  indicate  that  the  DOD  guidance 
and  policy  documents  needed  to  implement  and  manage  a 
software  safety  program  are  adequate.  Software  safety 
programs,  however,  are  not  Implemented  and  managed 
effectively;  in  addition,  structured  software  safety 
analysis  techniques  are  not  widely  used. 

To  improve  the  DOD's  ability  to  manage  weapon  system 
safety  programs,  DOD  should  implement  the  proposed  training 
program. 
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AN  ASSESSMENT  OF  SOFTWARE  SAFETY 


AS  APPLIED  TO  THE  DEPARTMENT  OF  DEFENSE 
SOFTWARE  DEVELOPMENT  PROCESS 

1 .  Introduction 


fiintral-liim 

Software  safety  la  an  important  leaue  throughout  the 
Department  of  Defense  (DOD)  In  the  acquisition  of  high 
technology  weapon  systems.  As  weapon  system  procurement 
approaches  the  twenty-first  century,  weapon  systems  are 
becoming  Increasingly  expensive,  complex,  and  computer 
dependant.  As  evidenced  in  the  Persian  Gulf  War,  the  United 
States  military  and  its  allies  rely  heavily  on  computers  to 
control  oritioal  decision  making  processes  in  weapon 
systems.  Increasing  dependence  on  computers  to  control 
safety-critical  functions  is  also  apparent  throughout 
Industry  in  areas  such  as  air  traffic  control,  mass  transit, 
ship  navigation,  and  nuclear  power  plant  operation. 

Many  documented  mishaps  directly  resulting  from 
computer  input  or  control  problems  can  be  found  in  the 
extensive  compilation  of  computer  related  mishaps  listed  In 
references  (37:6-7)  and  (38txi-xili) .  DOD  weapon  system 
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developnant  ha»  also  resulted  In  a  number  of  computer 
related  mishaps.  For  example: 

•  A  potentially  life  threatening  problem  with  the 
Air  Force  F-16  fighter  aircraft  was  dlscov''  'ed 
during  simulation  testing.  When  flying  1;  <rted, 
F-16  flight  software  would  deadlock  over  ti.  j 
choice  of  a  left  roll  or  right  roll  to  return  the 
aircraft  to  level  flight  (34:2).  Also,  an  F-ie 
was  damaged  during  early  flight  testing  because 
the  computer  allowed  the  landing  gear  to  be  raised 
while  the  aircraft  was  still  on  the  ground.  (35:6) 

•  A  Navy  F-18  fighter  and  Its  pilot  were  almost  lost 
because  a  wing  mounted  missile  failed  to  separate 
from  the  wing  after  Ignition.  The  computer  opened 
the  missile  holding  clamp,  fired  the  missile,  and 
then  closed  the  clamp  before  the  missile  had 
developed  enough  thrust  to  leave  the  wing.  As  the 
missile  continued  to  thrust,  the  aircraft  lost 
20,000  feet  of  altitude  before  the  pilot  was  able 
to  regain  control  of  the  aircraft.  (36:6) 

•  The  Air  Force's  only  flying  prototype  of  the  F-22 
Advanced  Tactical  Fighter  crashed  due  to  a 
speculated  software  problem.  It  Is  hypothesized 
that  the  flight  control  computer  was  unable  to 
nova  aircraft  control  surfaces  fast  enough  to  keep 
up  with  the  pilot's  commands.  As  of  this  writing, 
the  mishap  is  still  under  Investigation.  (20: A3) 

•  The  Navy  has  Identified  software  problems  with  the 
F-14D  aircraft  development  effort.  A  number  of 
software  defects  caused  cockpit  displays  to  go 
blank  and  erroneous  data  to  be  supplied  to  the 
mission  computer.  (43:10) 

A  system  failure  due  to  software  errors  or  Inadequacies 
can  be  disastrous.  "When  computers  are  used  to  control 
safety-critical  processes,  there  Is  a  need  to  verify  that 
the  software  will  not  cause  or  contribute  to  an  accident" 
(4:377).  Ideally,  software  would  be  tested  for  every 
conceivable  Input  and  output  condition  before  being  released 
for  military  or  public  use.  Because  of  the  complexity  of 
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many  safety-critical  systems,  however,  testing  for  every 
possible  input  and  output  condition  is  not  technically  or 
economically  feasible  (44:61).  In  1986  the  DOD  spent  an 
estimated  $11.4  billion  on  software  development  and 
maintenance  (4:2-4).  In  1990  the  estimated  cost  for 
software  development  and  maintenance  was  $30  billion  or  10 
percent  of  the  national  defense  budget  (22:65).  The 
shrinking  defense  budget  cannot  continue  to  support  the 
increasing  cost  of  software  development  when  most  of  the 
money  spent  is  for  the  correction  of  errors  (41:48) . 

Sotoart,  fiflggtY 

Software  safety  is  that  Inherent  part  of  system  safety 
specifically  concerned  with  the  potential  safety  risks 
associated  with  the  software  portion  of  a  system.  The 
relatively  new  concept  of  software  safety,  however,  has 
become  a  concern  only  as  a  result  of  an  increasing 
dependency  on  computers  to  perform  functions  with 
potentially  critical  safety  impacts  (39:636).  Software 
safety  must  be  evaluated  "within  the  context  of  system 
safety"  (25:223).  Software  is  not  unsafe  when  considered  in 
isolation  and  is  only  an  indirect  contributor  to  mishaps 
(25:223).  In  a  system  context,  however,  the  likelihood  or 
risk  that  software  may  cause  or  contribute  to  a  mishap 
becomes  a  safety-critical  issue.  The  goal  of  software 
safety  is  to  "ensure  that  software  executes  within  the 
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systan  oontaxt  without  raaultlng  In  unaooaptabla  risk" 

(26:35) . 

System  safety  is  a  discipline  that  seeks  to  Identify, 
evaluate,  and  control  hazards  In  a  total  system  context.  A 
hazard  can  be  defined  as  "a  set  of  conditions  (i.e.,  a 
state)  that  can  lead  to  an  accident  given  certain 
environmental  [operating]  conditions"  (25:223)  or  mors 
simply  "a  condition  that  Is  prerequisite  to  a  mishap"  (8:2). 
Mishap  la  then  defined  as  "an  unplanned  event  or  series  of 
events  that  leads  to  an  unacceptable  loss  such  as  death, 
Injury,  Illness,  damage  to  or  loss  of  equipment  or  property" 
(8:2).  Some  safety  engineers  go  even  further  and  Include 
environmental  harm  as  an  unacceptable  loss  (26:35). 

The  degree  of  risk  associated  with  the  operation  of  a 
system  is  classified  In  terms  of  both  severity  and 
probability  of  occurrence.  Severity  is  a  function  of  the 
worst  possible  loss  aasooiated  with  a  hazard  and  probability 
of  occurrence  is  a  function  of  the  likelihood  that  a  hazard 
will  occur  and  lead  to  a  mishap  (25:223).  Therefore,  a 
software  system  Is  considered  safety-’orltloal  If  there  is 
some  degree  of  risk  associated  with  the  Incorrect  operation 
of  that  system.  Loss  of  human  life  and  severe  equipment 
damage  are  two  examples  of  unacceptable  risks. 

Software  safety  should  not  be  confused  with  software 
reliability  or  software  security.  While  safety  does  have  a 
relationship  to  both  of  these  disciplines,  there  are  some 
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distinct  dlffsrsnoss.  Rsllablllty  rsqulramsnts  for  a  system 
are  intended  to  ensure  the  system  operates  within  a 
predefined  failure  rate.  Safety  requirements,  on  the  other 
hand,  are  Intended  to  limit  the  system's  failures  to  only 
those  failures  determined  to  have  acceptable  risks.  "If  the 
design  is  unsafe  to  start  with,  no  degree  of  reliability  is 
going  to  make  it  safe"  (1:20).  security  requirements  for  a 
system  are  intended  to  prevent  system  failures  due  to 
unauthorised  access  and  actions.  Safety  requirements,  on 
the  other  hand,  are  Intended  to  prevent  system  failures  due 
to  Inadvertent  actions  (2ltl2). 

Software  Safety  Requirements  and  Analysis 

A  strong  emphasis  must  be  placed  on  develuping  software 
requirements  that  include  software  safety.  It  is  a  well 
established  fact  in  the  field  of  software  development  that 
the  vast  majority  of  errors,  including  safety-critical 
errors,  found  in  software  systems  can  be  directly  attributed 
to  some  misunderstanding  or  oversight  in  requirements 
definition  and  design  activities  (40sl6,  2:1).  Therefore, 
the  roots  of  software  safety  must  form  early  in  the 
requirements  analysis  phase  of  the  software  development 
process  and  maintain  a  significant  level  of  managerial  and 
technical  support  throughout  the  life  of  the  system. 

Software  safety  analysis  is  the  principal  means  of 
evaluating  system  specif io  software  requirements.  In 
addition,  software  safety  analysis  can  provide  program 
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managers  some  level  of  oonfidenoe  that  their  software  will 
operate  at  an  acceptable  level  of  risk.  A  number  of 
structured  safety  analysis  techniques  have  been  specifically 
developed  or  adapted  for  performing  software  safety 
analyses.  These  techniques,  or  methods,  can  be  applied  with 
some  rigor  to  produce  consistent  analysis  data.  Structured 
software  safety  aiialysis  techniques  must  be  applied  to  the 
software  that  will  control  all  weapon  systems  currently 
under  development  and  in  use  to  reduce  the  risk  that 
software  will  cause  a  system  failure  which  may  lead  to  loss 
of  life  or  damage  to  equipment. 

Software  Development  and  Safety  Program  Quidance 

Software  development  policy  within  the  DOD  begins  with 
DOO  Instruction  5000.2,  Defense  Acouisitien  Management 
Policies  and  Procedures.  This  document  specifies  that 
software  development  will  be  governed  primarily  by 
DOD-STD-2167A,  Defense  System  Software  Development.  DOD 
Instruction  5000.2  also  specifies  MIL~STD-882B,  System 
aafftty,,  Eragram  Riwirtmintl^  the  governing  document  for 
system  safety  in  defense  acquisition  programs.  There  are 
also  numerous  guidelines,  handbooks,  instructions,  and 
regulations  that  provide  additional  insight  into  DOD 
software  development  and  system  safety.  Appendix  B  provides 
a  list  of  software  development  and  system  safety  guidance 
and  policy  documents. 
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Th«  doouinents  dlsousBed  abov*  outllno  what  aafaty 
analysaa  should  ba  parfomad  at  spaolflo  points  In  a 
softwara  davalopnant  affort  and  provlda  details  on  what 
should  be  covered  In  each  of  the  analyses.  None  of  the 
available  guidance,  however,  adequately  addresBes  the  tools, 
techniques,  and  methodologies  required  to  have  a 
comprehensive  software  safety  program.  Comprehensive 
guidance  and  a  firm  understanding  of  software  safety  tools, 
techniques,  and  management  methodologies  are  necessary  for 
the  Implementation  and  management  of  an  effective  software 
safety  program.  A  program  manager  today  has  no  way  of 
realistically  and  accurately  assessing  the  risks  that 
software  contributes  to  the  overall  safety  of  a  system.  As 
systems  grow  more  complex  and  computer  dependent,  software 
safety  cannot  ba  Ignored. 

Spaolflo  Problem 

The  DOD  needi*  resear  oh  to  establish  a  comprehensive 
management  methodology  for  Implementing  software  safety 
programs.  The  components  of  a  software  safety  program  are 
the  DOD  software  development  process,  DOD  system  safety 
program  requirements,  and  current  software  safety  analysis 
technlqu>.ji,4.  This  thesis  discusses  the  Interrelationships  of 
the  throe  components  of  a  software  safety  program  and 
determines  the  current  state  of  software  safety  analysis  In 
weapon  system  development. 
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To  carry  out  th«  roBaaroh  daflnad  in  tha  last  saotlon, 
wa  dlvldad  tha  affort  Into  flva  objaotlvas: 

1.  To  detarmlna  where  within  the  DOD  software 
davelopnant  process  system  and  software  safety  requirements 
are  addressed  by  Bnalyslnq  the  software  development  process 
as  outlined  In  DOD-STD-2167A,  PtftMi  SYltiin,  SQftvart 
Pi\alQpmtnt. 

2.  To  determine  what  the  current  DOD  system  and 
software  safety  requirements  are  and  how  or  If  they  can  be 
easily  integrated  into  the  software  development  process 
through  an  analysis  of  MIL-STD~e82B,  System  Safety  Program 

Riouirtininti. 

3.  To  determine  how  or  if  the  above  software  safety 
requirements  can  be  met  through  an  Investigation  of  current 
software  safety  analysis  techniques. 

4.  To  determine  what  software  safety  analysis 
techniques  are  actually  used  In  DOD  weapon  systems 
development  and  what  succeases  or  failures  have  been 
realized  by  their  application. 

5.  Based  on  data  collected  In  meeting  objective  4,  the 
final  objective  of  this  research  was  to  examine  the  DOD 
System  Safety  Manager's  (S8M)  job  and  develop  a  training 
program  to  meet  the  SSM's  hardware  and  software  safety 
management  needs. 
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This  chapter  describes  the  research  methodology  used  to 
address  the  specific  problem  and  objectives  stated  In 
Chapter  I.  The  first  section  will  describe  the  research 
design  followed  by  a  discussion  of  the  data  collection 
process,  the  population  from  which  the  data  was  collected, 
and  finally,  the  data  analysis  process. 

Raigflggh  Ptalgn 

This  research  was  carried  out  in  two  sequential  phases. 
The  first  phase  consisted  of  two  steps  and  was  designed  to 
lay  the  foundation  for  the  study  by  determining  how  existing 
guidance  documents  and  software  safety  analysis  techniques 
nay  be  combined  to  develop  a  software  safety  program. 

Step  1  Involved  answering  the  following  three  questions t 

1.  Where  within  the  DOD  software  development  process 
are  system  and  software  safety  requirements 
addressed? 

2.  What  are  the  current  DOD  system  and  software 
safety  requirements? 

3.  How  can  the  software  safety  requirements  be  net? 
These  questions  correspond  to  research  objectives  1,  2,  and 
3  stated  in  Chapter  I. 

Step  2  involved  a  survey  of  aerospace  industry  and  DOD 
personnel  to  assess  the  current  state  of  software  safety 
within  DOD  weapon  system  development  and  determine  what 
software  safety  analysis  techniques  are  actually  being  used. 
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This  step  corresponds  to  research  objective  4  stated  In 
Chapter  I. 

The  second  phaoa  of  this  research  explored  training  and 
education  requirements  for  System  Safety  Managers  located  at 
Aeronautical  Systems  Center,  Wrlght-Patterson  AFB,  Ohio. 

This  phase  corresponds  to  researoh  objective  5  stated  In 
Chapter  I. 

Data,  CQiliQtlon 

The  data  collection  for  this  researoh  Involved  two 
phases.  Table  1  shows  a  matrix  of  research  objectives  and 
data  oolleotlon  methods,  in  phase  one,  we  gathered  data  to 
address  the  first  four  researoh  objectives.  The  first  throe 
objectives  were  fulfilled  primarily  by  literature  review; 
the  results  are  reported  In  Chapter  III.  The  fourth 
objective  was  fulfilled  by  conducting  a  telephone  survey; 
the  results  are  reported  In  Chapter  IV. 


Table  1.  Research  Objective  Methodology  Matrix 


Phaaa 

Raitaroh 

Objaotlv* 

Lltaratur* 

Review 

Telephone 

Survey 

Document 

Review 

Perionel 

Interview 

1 

1 

X 

2 

X 

3 

X 

X 

4 

X 

2 

3 

X 

X 
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Tha  second  phase  of  data  collootlon  speolfloally 
addressed  research  objective  s.  Data  collection  for  this 
objective  consisted  of  reviewing  system  safety  training 
material,  system  safety  guidance  and  policy  documents,  and 
weapon  system  hazard  analysis  reports.  In  addition,  SSMs 
were  interviewed  for  their  perceptions  of  the  system  safety 
manger's  job.  our  analysis  of  SSM  training  and  education  is 
presented  in  Chapter  V. 

Literature  Review.  DOD  guidance  and  requirements 
documents  served  as  the  principal  means  of  describing  and 
analyzing  the  DOD  software  development  process  and  the 
system  safety  program  requirements.  The  open  literature 
provided  the  principal  means  of  identifying  and  describing 
current  software  safety  analysis  teohnigues. 

Telephone Survey.  The  telephone  interview  was  selected 
as  our  survey  methodology.  Because  the  application  of 
structured  safety  analysis  teohnigues  to  software  is  a 
relatively  new  praotioe,  determining  the  extent  and 
effectiveness  of  their  use  required  screening  a  large  pool 
of  sources  within  the  aerospace  industry  and  DOD.  We  needed 
to  determine  quickly  the  true  subject  knowledge  held  by 
potential  participants,  c.  William  Emory,  in  Business 
RBUflrch.MftthQdl/  suggests  that  telephone  surveys  tend  to  be 
the  quickest  and  moat  economic  method  to  reach  and  screen  a 
large  pool  of  potential  participants  (ISiSlS).  While  the 
telephone  interview  does  limit  the  length  of  interviews  and 
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tha  complexity  of  quest lone,  it  has  the  additional  advantage 
of  reducing  interviewer  bias  (I5:33lx’332) . 

Interviews  were  conducted  as  seml-struotured,  focused 
Interviews.  We  developed  a  set  of  questions  designed  to 
guide  the  topical  direction  and  coverage  of  the  Interview. 
Focus  questions  were  developed  to  facilitate  identification 
of  subject  area  experts,  to  gain  a  perspective  on  the  state 
of  software  safety  analysis  within  DOD  software  development, 
and  to  keep  the  discussions  on  track.  Additional  questions 
were  Included  for  administrative  and  participant 
characterliatlon  purposes.  The  survey  questionnaire  Is 
Included  In  Appendix  D. 

The  unstructured  nature  of  the  Interviews  established 
an  open  and  flexible  atmosphere  and  allowed  us  to  probe  as 
deeply  as  necessary  to  gain  a  full  understanding  of  the 
topic.  Aurvey  participants  ware  briefed  at  the  start  of 
each  Interview  about  the  objectives  of  the  research.  With 
each  question,  participants  were  allowed  to  respond  In  an 
unstructured,  open-ended  format.  Specific  follow  up 
questions  were  composed  as  the  Interviews  progressed. 

Population 

The  target  population  for  this  study  consisted  of  all 
aerospace  Industry  and  DOD  representatives  that  could  be 
classified  as  one  or  more  of  the  following! 

•  Recognized  experts  In  software  safety  analysis 

•  System/software  safety  engineers  and/or  managers 
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*  Pro j act  anginaers  and/or  managara 

No  attempt  was  mada  to  datarmlna  tha  slza  of  this 
population,  but  it  probably  consists  of  thousands  of 
individuals.  Also,  no  upper  limit  was  set  on  the  number  of 
individuals  that  would  be  included  in  this  study. 

Our  goal  was  to  contact  as  many  Individuals  as  possible 
in  order  to  cover  a  fairly  broad  spectrum  within  the 
aerospace  industry  and  DOD.  Fifty-four  individuals  within 
the  target  population  were  contacted.  However,  only  23  were 
experienced  in  software  development  and/or  software  safety. 
These  individuals  became  the  sample  population.  A  list  of 
the  individuals  included  in  the  sample  population  is 
provided  in  Appendix  B. 

Pata  AnalYBlB 

Data  obtained  from  the  telephone  interviews  were  used 
to  describe  the  current  state  of  software  safety  analysis  as 
part  of  DOD  software  development  efforts.  The  data  ware 
onalyzed  for  three  kinds  of  information:  First,  is  software 
safety  important  to  system  and  software  development  efforts? 
Second,  what  software  safety  analysis  techniques  are 
employed  and  what  sucoess  or  failure  has  been  realized? 
Third,  what  benefits  and/or  drawbacks  are  associated  with 
software  safety  analysis  teohniqpies? 

To  determine  the  overall  importance  of  software  safety 
to  a  software  or  system  development  effort,  survey 
partioipantr  provided  subjective  assessments  of  the 
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importance  of  software  safety  In  their  organizations.  An 
important  part  of  these  assessments  was  whether  or  not 
software  safety  was  an  integral  part  of  each  organization's 
development  process.  This  information  helped  determine  the 
general  level  of  industry  and  DOD  awareness  of  software 
safety. 

Survey  participants  were  asked  what  software  safety 
analysis  techniques  were  used  hy  their  organizations. 
Responses  determined  which  techniques  were  actually  being 
used  and  identified  other  techniques  not  encountered  during 
initial  literature  review.  Many  analysis  techniques  were 
identified,  however,  only  three  had  been  developed  or 
adapted  specif ioally  for  software  safety  analysis.  Further 
literature  review  provided  descriptions  of  these  newly 
identified  structured  analysis  techniques. 

survey  participants  were  also  asked  to  recount  any 
successes  or  failures  realized  by  the  application  of 
structured  software  safety  analysis  teohnlqpjss.  success  was 
defined  as  the  application  of  an  analysis  technique  that 
resulted  in  a  documented  statistical  reduction  in  system 
risk.  Participants  were  asked  to  provide  their  own 
definition  of  failure.  Survey  responses  were  assessed  to 
determine  the  use  and  viability  of  structured  software 
safety  analysis  techniques. 

Survey  participants  ware  asked  to  assess  the  benefits 
or  drawbacks  associated  with  the  application  of  software 
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safety  analysis  techniques.  These  data  were  used  to 
Identify  strengths  and  weaknesses  of  the  analysis 
techniques,  potential  problems  In  the  management  of  software 
safety  programs,  and  potential  training  deficiencies 
encountered  by  safety  engineers  and  managers. 
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intrpdugtlQn 

This  literature  review  analyzes  the  DOD  software 
development  process  and  000  system  safety  program  and 
investigates  current  software  safety  analysis  teohniqtues. 
Figure  1  shows  the  three  components  of  a  software  safety 
program.  Our  analysis  of  the  literature  will  determine 
where  system  and  software  safety  requirements  are  addressed 
within  the  software  development  process ,  what  the  system  and 
software  safety  program  requirements  are,  and  how  the 
software  safety  analysis  requirements  are  met. 


Softw#r* 

O*v«loprrwnt  Proctts 
CX30-STO-S1B7A 


Sy»t*m  Saftty  Prooram 
MIL-STO-BSSB 


Soflwara  Sataty 
Anaiysia 

TtcAniquea 


WHERE;  ara  fiy»tam/a<ni'*ware 

WHAT  era  tha  ayatam/aof twara 

HOW  era  aoftwera  aafaty 

rtquiramanti  ■ddraiaad’’ 

•afsty  t*equlr«msnts? 

annlyiiB  raqulranwnta  mat'’ 

Figure  1.  Software  Safety  Program  Components 


Although  software  safety  applies  to  all  computer  controlled 
systems,  this  literature  review  focuses  only  on  software 
safety  as  it  applies  to  the  acquisition  of  weapon  systems 
within  the  DOD. 
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This  chapter  is  divided  Into  three  sections.  The  first 
section  analyzes  the  DOD  software  development  process  as 
outlined  In  DOD~STD-2167A,  Defense  System  Software 
Development.  Section  two  analyzes  the  DOD  system  safety 
program  as  outlined  In  MIL-STD~e82B,  System  Safety  Program 
Recfulraments.  Each  task  section  within  the  standard  Is 
discussed  as  it  relates  to  the  Implementation  and  management 
of  a  system  safety  program  and  the  software  development 
process.  seo1;lon  three  describes  current  software  safety 
analysis  techniques. 

pgp  SgftwflM  .Pfylopmant  .PgQQMi 

A  process  In  Its  simplest  terms  Is  a  "set  of  operations 
occurring  in  a  defi.nlte  sequence  that  operates  on  a  given 
input  and  converts  it  to  some  desired  output"  (16:299). 

Taken  a  step  further «  a  software  development  process  may  be 
defined  as  a  set  of  "software  engineering  activities, 
Including  technical  and  managerial  ones,  that  are  carried 
out  In  the  production  of  software"  (31:234). 

There  are  several  software  development  models  and 
methods  available  to  guide  a  software  development  project. 
This  research  focuses  on  only  one:  "the  classic  life  cycle 
paradigm  for  software  engineering,"  this  model  "is  the 
oldest  and  the  most  widely  used  paradigm  for  software 
engineering"  (40:20-21).  This  paradigm,  commonly  referred 
to  as  the  "Waterfall  Model"  takes  a  "sequential  approach  to 
software  development  that  begins  at  the  system  level  and 
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progresnes  through  analysis,  design,  coding,  testing,  and 
maintenance"  (40:20).  The  waterfall  model  has  been  adopted 
by  the  DOD  and  used  to  guide  the  software  development 
process  for  many  major  weapon  systems. 

Every  software  development  project  presents  a  unique 
set  of  circumstances  and  system  requirements  which  determine 
the  amount  of  detail  req^ilred  In  each  technical  and 
managerial  phase  of  Its  development.  "The  goal  of  the 
software  development  process"  therefore,  "Is  to  consistently 
produce  a  quality  product  .  .  (5:5-1).  DOD-STD-2167A  Is 

the  primary  guide  for  DOD  software  development  projects  and 
has  expanded  the  traditional  waterfall  model  Into  eight 
distinct  development  phases. 

The  eight  software  development  phases  are  shown  In 
Figure  2.  Although  the  process  must  stay  relatively 
constant,  the  development  phases  often  occur  concurrently 
(5:5-2).  The  first  phase,  system  Requirements 
Analysis/Design  and  the  last  phase.  System  Integration  and 
Testing  are  common  to  hardware  and  software.  There  are  six 
intermediate  phases  peculiar  to  software.  For  this  research 
all  eight  phases  are  discussed  as  they  relate  to  software 
development  and  safety.  During  the  software  development 
process,  development  activities  are  monitored  through  a 
series  of  formal  reviews  and  audits  as  described  In 
MIL-STD-1521B,  Technical  Reviews  and  Audits  for  Systems. 
Equipment,  and  Computer  Systems.  Figure  2  shows  how  the 
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Flgura  2.  Software  Davalopnant  Prooeaa  (5:5-2) 


fomal  ravlawa  and  audita  ralata  to  tha  eight  aoftwara 
devalopinant  phaaaa.  In  addition,  tha  figuro  ahowa  tha  three 
ayatam  haaelinea  uaed  to  maintain  configuration  control  of 
the  ayatam  undar  davalopmant.  The  aoronyma  for  each  review 
and  audit  aa  wall  aa  haaalina  daflnitiona  are  included  in 
Appendix  A. 

Tha  development  proceaa  for  a  weapon  ayatam  begina  by 
determining  ayatam  ragulramenta«  Concept  Exploration  and 
Damonatration/Validatlon  are  tha  firat  two  phaaea  of  tha 
ayatam  aoquialtion  cycle  which  may  be  conducted  to  aatabliah 
ayatam  level  raquiramenta  if  no  pravioua  ragulramants  exlat. 
Hardware  and  aoftwara  development  then  take  place  during  the 
Full  Scale  Development  phaae  of  the  ayatem  acguiaitlon 


19 


oyole.  Thtt  software  development  prooeae  actually  begins 
while  the  system  level  requirements  are  analyzed  and  an 
initial  allocation  of  system  functions  is  made  to  hardware 
and  software. 

System  Requirements  Analvsls/Deslan.  The  System 

Requirements  Analysls/Design  phase  refines  the  system  level 

requirements  and  establishes  the  system's  functional 

baseline  (5!5-4|  I7tl2).  Requirements  refinement  may  be 

aooompllahed  by  gathering  Information  through  independent 

analysis,  trade  studies,  and  prototype  development  (17:12). 

The  functional  bt  .dine  is  established  by  the  approval  of 

. . .  the  system  specification  and  the  operational 
concept;  by  developing  the  initial  subsyatem/segment 
designs;  and  by  refining  the  systems  engineering 
planning  activities  to  be  employed  during  system's 
development.  (St 5-4, 5-5) 

Expected  outputs  from  system  Requirements 

Analysis/Design  are 

•  System  Specifications 

•  System/Segment  Design  Documents 

•  Software  Requirements  Specification  (Preliminary) 

•  Interface  Requirements  Specification  (Preliminary) 

•  Operational  Concept  Document 

•  Software  Development  Plan 

«  Configuration  Management  Plan  (5:5-3). 

The  System  Requirements  Analysis/Design  phase  concludes 
with  a  System  Requirements  Review  (SRR)  and  a  System  Design 
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Review  (SDR).  At  the  SRR  the  developer's  progress  toward 
understanding  and  defining  the  system  level  requirements  Is 
evaluated  (5:5-2).  Two  Important  topics  discussed  at  this 
meeting  specifically  relating  to  software  safety  are  the 
program  risk  analysis  and  the  system  safety  analysis 
conducted  by  the  contractor  (9:19-20).  These  analyses  can 
provide  Important  information  concerning  the  potential 
hazards  and  risks  associated  with  the  design  and  operation 
of  t'.«i  system  under  development  and  should  be  used  to 
develop  requirements  to  eliminate  or  control  hazards  (9:21). 

At  the  SDR  all  system  level  requirements  are  reviewed 
In  preparation  for  establishing  the  functional  baseline 
(5:5-2).  Once  the  functional  baseline  Is  established,  the 
development  process  can  move  on  to  the  next  phase.  An 
Important  part  of  the  SDR  Is  the  review  of  failure-mode  and 
effects  analyses  to  ensure  identification  and  ranking  of 
technical  and  program  risks  (9:24).  In  addition,  system 
hazard  analyses  are  reviewed  with  an  emphasis  on 
Identification  of  safety  test  requirements  (9:25).  The 
results  of  the  developer's  analyses  become  an  Important  part 
of  the  program  manager's  decision  to  progress  to  the  next 
phase  of  development. 

Anftlyiii »  The  software 

Requirements  Analysis  phase  establishes  "detailed 
functional,  performance,  interface,  and  qualification 
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r«qulrttm«nt8  for  oaoh  CSCI  [Conputar  Softwar*  Configuration 
Itam]  basad  on  tha  Syatan  Spaoifloatlon"  (5:5-6).  This 
phase  also  identifies  system  level  testing  requirements 
(5:5-6) . 

Expected  outputs  from  Software  Requirements  Analysis 
are 

•  Software  Requirements  Speolfloatlon 

•  Interface  Requirements  Specification 

•  Software  Development  Plan  (Updated)  (5:5-6). 

The  Software  Reqpalrements  Analysis  phase  concludes  with 
a  Software  Specification  Review  (SSR) .  The  SSR  covers  each 
CSCI's  software  requirements  as  outlined  in  the  Software 
Requirements  specification  and  the  Interface  Requirements 
Specification  (9:31).  This  review  also  provides  another 
opportunity  for  ris)c  assessment  in  regards  to  software 
safety  by  examining  updates  to  previously  delivered  safety 
data  (9:31-32).  Successful  completion  of  the  SSR 
establishes  the  allocated  baseline. 

Preliminary  Design.  The  Preliminary  Design  transforms 
system  and  software  requirements  into  some  type  of  software 
architecture  (40:215).  During  this  phase  the  contractor 
uses  the  outputs  from  the  previous  development  phases  and 
"partltion[s]  the  software  into  components  and  defina[s]  the 
function  of  each  component  and  the  relationships  between 
them"  (5:5-7). 
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Expaotad  outputs  from  Pralimlnary  Doslgn  ara 

•  Softwara  Daaign  Dooumant  (Prallmlnary) 

•  Softwara  Taat  Plan  (Taat  Idantlfioatlon) 

•  Intarfaca  Daaign  Dooumant  (Prallmlnary)  (5:5-7). 
Tha  Prallmlnary  Daaign  phaaa  oonoludaa  with  a 

Prallmlnary  Daaign  Ravlaw  (PDR) .  Tha  PDR  la  a  oomprahanalva 
ravlaw  of  tha  antlra  ayatam  and  provldaa  tha  flrat  In-dapth 
ravlaw  of  ayatam  aafaty  laauaa.  Spaolflo  aafaty  Itama  to  ba 
ravlawad  ara 

•  raaulta  of  qualltatlva  and  quant Itatlva  hazard 
analyaaa ; 

•  raaulta  of  ayatam  and  Intra-ayataxn  aafaty 
Intarfaoa  trada-off  atudlaa/ 

•  aafaty  raqulramanta  paaaad  down  to  auboontraotora ; 

•  apaolal  or  ayatam  paoullar  aafaty  araaar 

•  adaquaoy  of  prallmlnary  daaign  for  maatlng  aafaty 
raqulramanta  (9:42). 

Tha  PDR  aaaaaaaa  tha  faaslblllty  of  tha  davalopar'a  daaign, 
aaaaaaaa  currant  rlaka,  and  datarmlnaa  whathar  tha 
davalopmant  affort  la  raady  to  prograaa  Into  tha  naxt  phaaa 
(9:33) . 

Datallad  Dealon.  "Tha  purpoaa  of  tha  Datallad  Daaign 
activity  la  to  logically  daflna  and  oomplata  a  aoftwara 
daaign  that  aatiaflaa  tha  allooatad  raqulramanta"  (5:5-8). 
During  this  phaaa,  tha  davalopar  raflnaa  tha  prallmlnary 
daaign  to  a  laval  of  datall  that  allows  davalopmant  of  tha 
oomputar  program  by  somaona  othar  than  tha  daslgnar  (5:5-8). 
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If  thar*  art  any  davlatlona  from  tha  softwara  raqulramanta 
apaolfloatlon,  thay  should  ba  dafinad,  juatlflad,  and 
aubmlttad  to  tha  govarnroant  for  approval. 

Expaotad  outputs  from  Data 11 ad  Daalgn  ara 

•  Softwara  Daalgn  Dooumant  (Datallad  Daalgn) 

•  Softwara  Taat  Daaorlptlon  (Caaaa) 

•  Intarfaoa  Daalgn  Dooumant  (StB-8). 

Tha  Datallad  Daalgn  phaaa  oonoludaa  with  a  Critical 
Daalgn  Ravlaw  (CDR) .  Tha  CDR  "aaaura[a]  that  tha  aoftwara 
daalgn  aatlaflaa  tha  raqulramanta  of  both  tha  ayatam  laval 
apaolfloatlon  and  tha  aoftwara  davalopmant  apaoifloatlon*' 
(5iS-9}.  Syatam  aafaty  Itama  of  particular  oonoarn  at  tha 
CDR  Inoluda 

•  ravlaw  of  datallad  daalgn  for  aafaty  daalgn 
raqulramant  oompllanoa; 

«  ravlaw  aooaptanoa  taat  raqulramanta  for  Inolualon 
of  adaquata  aafaty  raqulramanta; 

•  ravlaw  oparatlonal  malntananoa  aafaty;  analyaaa 
and  prooaduraa  (9t61). 

SuGoasaful  oomplatlon  of  CDR  anablaa  tha  davalopar  to  bagln 
tha  naxt  phaaa. 

fittdiag-ftnd  ,,.TiM.tlng »  coding 

tranalataa  tha  datallad  aoftwara  daalgn  Into  a  apaolflo 
programming  language  (5t5-9).  Souroa  coda  llatlnga  ara 
ganaratad  aa  tha  program  la  oomplatad.  It  la  tha 
programmer 'a  raaponalblllty  to  examine  oomplatad  Computer 
Softwara  Unit  (CSU)  code  for  arrora  (5: 5-9).  If  oonaldarad 
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•rror  fr««  and  oapabla  of  maatlng  raqfulrananta,  tha  code  la 
oompllad  and  an  objaot  ooda  listing  la  obtalnad. 

Aftar  coding,  but  bafora  compilation,  aaoh  CSU  must  be 
tasted  to  eliminate  any  errors  that  may  exist  (5:5->9). 
Particular  attention  la  paid  to  tha  algorithm  and  logic 
Implamantad  as  wall  as  to  tha  CSU's  ability  to  satisfy 
raqulramants  (7i27).  Spaolfio  informal  test  procedures  are 
generated  and  recorded  In  the  csu*s  Software  Development 
Folder  (SDF)  along  with  the  results  of  testing.  If  errors 
are  found,  the  software  requirements  and  design  documents 
might  be  revised  and  the  CSU  Is  re»ooded  and  re-tested. 
Finally,  test  procedures  are  developed  for  testing  the 
Computer  Software  Component  (CSC)  that  a  particular  CSU  Is 
part  of  (7127).  These  CSC  test  procedures  are  recorded  In 
the  CSC's  development  folder. 

Expected  outputs  from  CSU  Coding  and  Testing  are 

«  CSU  source  and  object  code  listings 

•  CSU  test  procedures  and  results 

•  CSC  test  procedures  (7t27). 

No  formal  review  Is  associated  with  the  Code  and  CSU 
Testing  phase.  CSU  coding  and  testing  leads  directly  to 
Computer  Software  Component  Integration  and  Testing. 

fioBiputtr  ggftwart  ttompQntnti  Tasting. 

Computer  Software  Component  integration  and  Testing  Is 
conducted  to  "combine  the  software  units  and  components  that 
have  been  Independently  tested  Into  the  total  software 
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product  and  to  damonatrata  that  thla  combination  fulfllla 
tha  ayatam  daalgn"  (5:5*10).  CSUa  ara  Intagratad  into  CSCs 
which  in  turn  may  be  integrated  into  larger  CSCa.  Each 
combination  la  teatad  to  ensure  that  the  implementation 
algorithm  and  logic  are  error  free  and  that  raqulramanta  arc 
aatlafiad  (7:29).  Teat  raaulta  determine  whether 
requirement a  and  daalgn  dooumenta  need  to  be  updated  and 
CSUa  re-coded  and  re*teated  (7:29).  In  preparation  for 
Formal  Qualification  Testing  (FQT) ,  each  teat  oaaa  in  the 
Software  Test  Description  (STD)  ahould  be  pretested  during 
thia  phase  (7:29).  Teat  procedures  for  CSClt  testing  are 
also  developed  at  thla  time. 

Expected  outputs  from  CSC  Integration  and  Testing  are 

•  Software  Teat  Description  (Procedures) 

•  CSC  test  results  (7:29). 

The  CSC  Integration  and  Testing  phase  concludes  with  a 
Teat  Readiness  Review  (TRR)  (5:5*10).  At  the  TRR,  the 
results  of  informal  testing  are  reviewed  and  software  teat 
procedures,  including  safety  test  procedures,  ara  evaluated 
for  adeqruaoy  and  compliance  with  test  plana  (9:69).  The  TRR 
la  used  to  ascertain  whether  or  not  the  development  la  ready 
to  progress  to  CSCI  Testing  (9:69). 

Computer  Software  Configuration  Item  Testing.  CSCI 
Testing  is  conducted  to  test  each  CSCI  using  the  software 
teat  plans  and  procedures  written  earlier  in  the 
development.  CSCI  testing  constitutes  Formal  Qualification 
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T«Btln9  and  servaa  to  •atablish  the  product  baseline  (9:31). 
Each  CSCl  requirement  In  the  Software  Requirement 
Specification  and  Interface  Requirement  Specification  must 
be  verified.  Results  of  testing  are  recorded  In  Software 
Test  Reports  and  any  errors  found  are  analysed  for  their 
system  and  safety  Impacts,  if  deemed  necessary/ 
requirements  and  design  documents  are  updated  and  CSUa, 

CSCs,  and  CSCIs  are  re-coded  and  re-tested  (9:31). 

Expected  outputs  from  CSCI  Testing  are 

•  Software  Test  Reports 

•  [Final]  Software  Product  Specification 

•  [Final]  Source  Code  Listings 

»  [Final]  Software  Design  Document  (5:5-11). 

By  completion  of  CSCI  Testing,  the  Software  Product 
Specification  (SPS)  and  all  support  documents  necessary  to 
operate  and  maintain  the  fielded  software  product  are 
finalized.  Support  documents  Include 

•  Computer  System  Operator's  Manual 

•  Software  User's  Manual 

•  Software  Programmer's  Manual 

•  Firmware  support  Manual 

•  Computer  Resources  Integrated  Support  Document 

•  Version  Description  Document  (5:5-ll). 

The  CSCI  Testing  phase  typically  concludes  with  the 
Functional  Configuration  Audit  (FCA)  followed  by  the 
Physical  Configuration  Audit  (PCA)  (5:5-11).  The  FCA  and 
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PCA  nay,  howevor,  be  run  oonourrently  and  are  aonetines 
deferred  until  after  System  Integration  and  Testing.  The 
FCA  verifies  that  each  CSCI  performs  In  accordance  with  Its 
requirements  and  Interface  specification  (9:69).  The 
software  test  reports  are  examined  and  the  appropriate 
operations  and  support  documents  are  reviewed  (StS-ll).  The 
PCA  validates  the  finished  software  product  against  Its 
design  documentation  (9i75).  The  audit  involves  direct 
comparison  and  examination  of  the  SPS  and  the  source  code 
listing.  Successful  completion  of  the  PCA  establishes  the 
product  baseline  against  which  all  subseqtuent  changes  must 
be  made  by  formal  engineering  change  proposals  (9:75). 

SYSttm  IntdqrfttlQn  and-Xuliinq.  The  system  integration 
and  Testing  phase  ensures  that  the  newly  developed  software 
will  work  with  the  overall  system  In  an  operational 
environment  (Si5>l2).  The  hardware  and  software  performance 
as  well  as  system  Interfaces  are  examined  and  compared  to 
system-level  specifications  (9:83).  Often,  the  FCA  and  PCA 
are  conducted  during  this  phase. 

The  expected  outputs  of  System  Integration  and  Testing 
are  any  final  updates  to  requirements,  design,  and  interface 
documents.  In  addition,  all  documentation  and  source  and 
object  code  listings  should  be  delivered  (7:33). 

The  system  Integration  and  Testing  phase  ends  with  a 
Formal  Qualification  Review  (FQR) .  "The  FQR  Is  a 
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sy8t«m-l«vel  r«vl«w  that  varlfiaa  that  the  actual  system 
psrformanoe  oompllss  with  system  [Including  safety] 
requirements"  (5:5-12). 

Software  Development  Process  Summary.  DOD-STD-2167A 
defines  a  detailed,  highly  structured  software  development 
process.  The  process  description  emphasizes  process  steps 
and  their  expected  outputs  In  the  form  of  formal  documents 
and  software  products.  Program  managers  have  the  authority 
to  tailor  the  process  to  fit  the  peculiar  needs  of  their 
program.  Although  D0D-STD-2167A  makes  a  few  references  to 
system  safety,  It  does  not  specifically  address  software 
safety. 

MZL-STD-1521B  also  defines  detailed  and  highly 
structured  reviews  and  audits,  but  unlike  DOD-STD-2167A, 
MIL-STD-1521B  discusses  In  detail  those  safety  items  that 
should  be  reviewed  at  specific  points  In  the  development 
process.  Each  formal  review  and  audit  Indicates  specific 
system  and  software  safety  Items  for  review.  Program 
managers,  however,  can  limit  the  specific  items  to  be 
discussed  at  any  review  or  audit. 

This  analysis  of  the  software  development  process  along 
with  Its  formal  reviews  and  audits  reveals  significant 
emphasis  on  system  and  software  safety.  The  principal 
limitation  of  the  development  process,  from  a  safety 
standpoint,  is  that  potentially  all  of  the  safety  review  can 
be  tailored  out  of  the  process. 
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A  systen  safety  program  Is  an  essential  element  In  the 
DOD*s  acquisition  of  high  technology  weapon  systems.  The 
primary  document  currently  In  use  by  the  DOD  to  Impose 
requirements  for  developing  and  Implementing  a  system  safety 
program  Is  MIL‘‘STD-882B  with  Notioe  1.  According  to 
MIL-STD-882B: 

The  principal  objective  of  a  system  safety  program 
within  the  Department  of  Defense  (DOD)  la  to  make  sure 
safety,  consistent  with  mission  requirements,  is 
designed  into  systems,  subsystems,  equipment,  and 
facilities,  and  interfaces.  (8:lil) 

To  accomplish  this  objective,  MIL-STD-882B  provides 

general  system  safety  program  guidance,  definitions,  and 

requirements  as  well  as  specific  system  safety  tasks 

contained  in  three  task  sections:  Task  Section  100,  Task 

Section  200,  and  Task  Section  300.  Bach  task  section 

consists  of  several  specific  tasks  which  are  selectively 

applied  to  a  development  effort.  Tasks  may  be  used  as 

written  or  tailored  to  meet  the  specific  needs  of  the 

program.  The  degree  to  which  these  tasks  are  levied  against 

the  system  development  contractor  is  the  responsibility  of 

the  System  Safety  Manager  (SSM)  (8:3). 

System  Safety  Program  Requirements.  A  totally  safe 

system  is  often  not  an  affordable  option  in  a  major  weapon 

system  acquisition.  Therefore,  a  system  is  usually  designed 

to  an  acceptable  degree  of  safety.  This  degree  of  safety 

implies  some  inherent  risk  in  the  system.  The  emphasis 
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within  system  safety  is  to  identify  and  analyze  potential 
system  hazards  to  determine  what  level  of  risk  each  poses. 
Efforts  are  then  made  to  eliminate  or  reduce  these  risks. 

It  is  the  program  manager's  responsibility  to  specify  the 
acceptable  level  of  risk  for  a  program. 

Identified  hazards  are  controlled  systematically.  The 
first  step  is  to  eliminate  the  hazard  or  reduce  its 
associated  risk  through  design.  Next,  if  the  hazard  cannot 
be  designed  out,  it  must  be  controlled  by  designing  and 
installing  safety  devices  or  it  should  be  isolated  via  the 
design  and  installation  of  protective  systems.  The  third 
step  is  to  provide  systems  and  devices  that  detect  the 
hazard  condition  and  produce  a  warning  signal.  Finally,  for 
those  hazards  that  can  neither  be  eliminated  nor  have  their 
associated  risk  reduced,  special  procedures  and  training  to 
implement  the  procedures  must  be  developed  and  included  in 
technical  manuals  (8:6). 

In  order  to  assess  the  risk  of  any  hazard,  the  hazard 

must  be  categorized  in  terms  of  its  severity  and  pro'  -Ability 

of  occurrence.  KIL*-STD>882B  defines  hazard  severity  as 

...  a  qualitative  measure  of  the  worst  credible  mishap 
resulting  from  personnel  error;  environmental 
conditions;  design  inadequacies;  procedural 
deficiencies;  or  system,  subsystem  or  component  failure 
or  malfunction  ....  (8:6-7) 

Table  2  shows  some  sample  hazard  severity  categories  that 
may  be  used  by  program  managers  as  guidance  when  defining 
severity  categories  for  their  system. 
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Tabltt  2.  Hazard  Savarity  Catagorlas  (8:7) 


1  Category 

Description 

Mishap  Definition 

l-l 

CATASTROPHIC 

Death  or  system  loss. 

II 

CRITICAL 

Severe  injury,  severe  occupational 
illness,  or  major  system  damage. 

III 

MARGINAL 

Minor  injury,  minor  occupational 
illnoas,  or  minor  system  damage, 

IV 

NEGLIGIBLE 

Less  than  minor  injury,  occupational 
illness,  or  system  damage, 

Hazard  probability  lb  ^wflnad  as  "tha  probability  that 
a  hazard  will  ba  oraatad  during  tha  planned  Ufa  axpaotanoy 
of  tha  ayatan  .  .  (8:7).  Potantlal  hazard  ooourranoaa 

oan  ba  naaaurad  par  unlta  of  tlna  or  by  othar  avanta  and 
catagorlaa  of  Intaraat.  Hazard  probabllltlaa  ara  fairly 
aaay  to  naaaura  In  a  nature  ayatan  In  which  taatlng  or 
operation  haa  begun.  Early  In  a  progran'a  davalopnant, 
however,  hazard  probabllltlaa  nuat  ba  obtained  through  a 
combination  of  raaearch  and  analyala,  to  Include  the 
evaluation  of  hlatorlcal  data.  Table  3  ahowa  aone  sample 
hazard  probability  levels.  Again,  the  program  manager  nuat 
determine  acceptable  or  unaooaptable  hazard  probability 
rankings. 

It  la  Important  to  note  that  tha  general  system  safety 
program  requirements  do  not  differentiate  between  hardware 
and  software.  The  raqulranents  emphasis  la  on  a  safe 
system.  T***^  task  sections  of  NIL<-STD-882B  provide  the  means 
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Table  3.  Hazard  Probability  Rankings  (8:7) 


Laval 

Dascrlptlon 

Specific  Individual 
Item 

Fleet  or  Inventory 

B 

FREQUENT 

Likely  to  occur 
frequently 

Continuously 

experienced 

B 

PROBABLE 

Will  occur  aoveral 
tlmoa  In  life  of  an 

Item 

Will  occur 
frequently 

C 

OCCASIONAL 

Likely  to  occur 

Bomatina  In  life  of  an 
item 

Will  occur  aeveral 
times 

D 

REMOTE 

Unlikely  but  posalble 
to  occur  In  life  of  an 
Item 

Unlikely  but  can 
reasonably  be 
expected  to  occur 

E 

IMPROBABLE 

So  unllkaly,  It  can  be 
aBiunad  occurrence  may 
not  be  experienced 

Unlikely  to  occur, 
but  possible 

to  tailor  a  system  safety  program  for  both  the  hardware  and 
software  of  the  system  being  developed. 

Task  Seotion  IQQ.  Task  Section  100  provides  speoifio 
program  management  and  control  tasks  that  may  be  employed  to 
establish  a  system  safety  program  for  a  system  development 
effort.  Table  4  lists  the  100  series  tasks.  In  addition  to 
the  task  listings,  Appendix  A  of  MIL~STD-882B  provides 
guidance  to  the  SSM  in  the  task  selection  and  implementation 
(for  100,  200,  and  300  series  tasks)  required  to  establish 
and  manage  a  system  safety  program. 

From  a  software  safety  perspective.  Task  101,  System 
Safety  Program  Plan,  is  the  most  Important  of  the  lOO  series 
tasks.  This  task  requires  the  developer  to  describe  his 
approach  to  system  safety.  Specific  procedures  and  analysis 
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Tabl0  4.  Program  Managamont  and  Control  Tasks 


Task  Number 

Title 

100 

System  Safety  Program 

101 

System  Safety  Program  Plan 

102 

Integration/Managemcnt  of  Associate  Contractors) 
Subcontractors,  and  Architect  and  Engineering  Firms 

103 

System  Safety  Program  Reviews 

104 

System  Safety  Oroup/System  Safety  Working  Group  Support 

105 

Hazard  Tracking  and  Risk  Rssolution 

106 

Test  and  Evaluation  Safsty 

107 

System  Safety  Progress  Summary 

loe 

Qualifications  of  Key  Contractor  System  Safety 
Engineerii/Msnagera 

msthodologiss  ars  prsssntsd  to  ths  program  offlos  for  rsvlsw 
and  approval.  Ths  Systsm  Safsty  Program  Plan  providas 
Information  on  tha  davalopar's  ooncaptuallzatlon  of  tha 
hardwara  and  softwara  ralationshlps  In  a  systam  and  how 
softwara  aafaty  will  ba  traatad  in  ralatlonship  to  system 
safety. 

Task  saotlon  2QQ.  Task  Section  200  providas  specific 
design  and  evaluation  tasks.  Tha  200  series  tasks  are 
designed  to  provide  guidance  for  hardwara  safety  analyses. 
However,  these  tasks  may  be  used  to  assure  some  level  of 
software  safety  in  systems  where  software  plays  a  minor  role 
and  separate  software  analyses  are  not  economical  (1:21). 
Table  S  lists  the  200  series  tasks. 
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Tabl«  5.  Daslgn  and  Evaluation  Tasks 


Task  Number 

Title 

201 

Preliminary  Hazard  List 

202 

Preliminary  Hazard  Arialysis 

203 

Subsystem  Hazard  Analysis 

204 

System  Hazard  Analysis 

203 

Operating  and  Support  Hazard  Analysis 

206 

Oooupatlonal  Health  Hazard  Assessment 

207 

Safety  Verification 

206 

Training 

209 

Safety  Assessment 

210 

Safety  Compliance  Assessment 

211 

Safety  Review  of  Engineering  Change  Proposals  and 
Requests  for  Devlatlon/Valver 

212 

Reserved 

213 

QFE/OFP  System  Safety  Analysis 

Tha  Initial  phasas  of  a  aystan  davaloprnant  Involva 
avaluatlng  thosa  hazards  that  a  systam  might  anoountar  so 
that  tha  ramalnlng  systam  safaty  work  affort  can  ba  daflnad 
(8:A-ll).  Task  201,  Prallmlnary  Hazard  List  (PHL) ,  and  Task 
202,  Prallmlnary  Hazard  Analysis  (PHA) ,  call  for  a 
prallmlnary  look  at  tha  antlra  systam  Involving  both 
hardwara  and  softwara.  Thorough  PHLs  and  PHAs  form  tha 
"framawork  for  othar  hazard  analysas  whloh  may  ba  parformad" 
(BiA-ll).  For  axampla,  tha  PHL  and  PHA  form  tha  foundation 
for  subsaquant  Implamantatlon  of  Task  301,  Softwara 
Raquiramants  Hazard  Analysis. 
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Th«  PHL  Ig  mgraly  a  compilation  of  poasibla  hazards 
takan  from  historical  rsoords  and  soma  brainstorming  by  the 
analysts.  An  Initial  datarminatlon  of  hazard  significance 
is  also  made.  The  PHA  is  a  continuation  of  the  PHL.  It 
identifies  safety  critical  areas  in  the  system  through  an 
initial  hazard  evaluation,  and  identifies  safety  design 
criteria,  specific  system  concerns  for  a  PHA  include 
hazardous  components;  safety  related  interfaces  (includes 
software  controls);  environmental  constraints;  operating, 
teat,  maintenanoe  and  emergency  procedures;  facilities  and 
support  equipment;  and  safety  equipment  (8:201-1,  202-1). 

The  next  three  tasks  are  all  extensions  of  the  PHA. 

Task  203,  Subsystem  Hazard  Analysis  (SSHA) ,  extends  the  PHA 
to  identify  hazards  associated  with  the  design  of 
subsystems.  Task  204,  system  Hazard  Analysis  (SHA) 
analyzes  system  level  hazards  including  subsystem 
interfaces.  Task  205,  Operating  and  Support  Hazard  Analysis 
(O&SHA) ,  analyzes  operating  and  support  hazards  (8:203-1, 
204-1,  205-1). 

Another  task  with  potential  software  safety 
implications  is  Task  207,  Safety  Verification.  The  purpose 
of  this  task  is  "to  define  and  perform  tests  and 
demonstrations  or  use  other  verification  methods  on  safety 
critical  hardware,  software,  and  procedures  ..." 

(8:207-1).  This  task  allows  verification  of  the  successful 
elimination  or  control  of  safety  hazards. 
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Ta«k  Saotlon  300.  Th«  300  ■•rl«s  tasko  "apaolfy  tha 
typaa  of  aoftvara  aafaty  analyaaa  that  ahould  ba  parformad 
aa  part  of  a  oomprahanalva  aafaty  program"  (1:21).  Tha  300 
aarlaa  taaka  ara  llatad  In  Tabla  6.  Thaaa  taaka;  added  In 
July  1987  and  llatad  In  Hotloa  1  to  MIL-STD-882B>  oomplamant 
tha  aoftwara  davalopmant  procaaa  aa  outllnad  In 
DOD-8TD-2167A  (1:21).  Figura  3  ahowa  how  thaaa  taaka 
oorraapond  to  tha  aoftwara  davalopmant  prooaaa.  Tha  300 
aarlaa  taaka  wara  davalopad  to  provlda  inoraaaad  amphaala  on 
aoftwara  aafaty  anglnaerlng  funotlona  and  analyala 


throughout  tha  antlra  aoftwara  davalopmant  prooaaa  (32:27). 


Tabla  6.  Softwara  Haaard  Analyala  Taaka 


Talk  Nutnbir 

Tltla 

301 

Softwara  Raquiraroanti  Hazard  Analyala 

302 

Top-Laval  Daalgn  Hazard  Analyila 

303 

Datallad  Daalgn  Hazard  Analyala 

304 

Coda-Lava 1  Softwara  Hazard  Analyala 

305 

Softwara  Safaty  Taatlng 

306 

Softwara/Uaar  Intarfaca  Taatlng 

307 

Softwara  Changa  Hazard  Analyala 

Tha  Softwara  Raqulramanta  Hazard  Analyala  (SRHA) ,  Taak 
301,  la  tha  flrat  and  parhapa  tha  moat  Important  taak 
apaolfloally  daalgnad  to  Inaura  aoma  laval  of  aoftwara 
aafaty  within  a  ayatam.  Ita  purpoaa  "la  to  davalop  aafaty 
daalgn  raqulramanta  to  ba  Inoludad  In  tha  ayatam  and 
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rigurt  3.  R«latlonthip  batwaan  300  sarlaa  Taa)ca  and 
tha  Softwara  Davalopmant  Prooaaa  (It 22) 


aoftwara  pralltnlnary  daalgn"  (li22).  To  naat  tha 
prallminary  daalgn  daadllna,  tha  SRHA  ayatam  aafaty 
raquiranant  muat  ba  lavlad  againat  tha  davalopar  aarly  In 
tha  Syatom  Raqulrananta  Analyaia/ Daalgn  phaaa  of  tha 
davalopnant  prooaaa. 

Baalo  Inputa  Into  tha  SRHA  ara  tha  rlaka  Idantll’lad 
through  tha  Prallminary  Hazard  Llat  (Taak  201) ,  and  tha 
Prallminary  Hazard  Analyaia  (Taak  202)  (B:30l-l}.  outputs 
from  tha  SRHA  ara  uaad  to  build  tha  foundation  for  ayatam 
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levcil  safaty  raqulramanta  and  aarva  aa  Inputa  to  tha 
auooaadlng  300  aarlaa  aafaty  analyaaa  (lt22,  8:301-1). 

Taaka  302  through  306  parallel  different  phases  of  the 
davalopnant  prooaas  and  are  designed  to  provide  apeuifio 
safety  analyses  within  each  phase.  Although  each  analysis 
builds  on  the  information  obtained  in  the  SRHA,  eaoh  also 
relies  on  different  input  documents  and  the  required  level 
of  detail  within  eaoh  task  grows  more  intense  as  the  system 
approaches  the  final  stages  of  development  (1:22-24) . 

The  Software  Change  Hazard  Analysis  (SCHA) ,  Task  307, 
has  two  functions  within  the  system  safety  process.  First, 
it  requires  the  developing  contractor  to  analyze  eaoh  and 
every  safety-oritioal  design  change  made  during  the 
development  proosss  to  insure  system  safety  is  not 
inadvertently  oompromised  (1:24).  The  second  function  of 
the  SCHA  begins  onoe  the  system  under  development  is 
delivered  to  the  user.  At  this  time  the  SCHA  becomes  an 
extension  of  all  the  300  series  tasks  and  provides  a  means 
in  which  to  continue  the  software  safety  process  throughout 
the  software  support  phase  of  the  system  (1:24). 

fifyatia .  aagihY. ,  ErcgiAza,,  aummagy. «  nil-8td-882b  with 
Notice  1  provides  the  current  guidanoe  for  system  safety 
requirements  in  weapon  system  acquisitions.  The  general 
system  safety  program  requirements  included  in  NIL-STD-8B2B 
emphasize  developing  a  safe  system.  They  do  not 
differentiate  between  hardware  and  software,  specific 
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«af«ty  racaulramants  are  provided  by  the  100,  200,  and  300 
aerlee  taeke. 

The  100  series  tasks  provide  systan  safety  management 
guidance  and  are  the  most  Important  in  establishing  a  system 
safety  program.  The  200  series  tasks  are  geared  toward  the 
reduction  of  hardware  hazards,  but  are  often  used  to  include 
software.  The  300  series  tasks  complement  the  DOD  software 
development  prooess  and  represent  an  attempt  by  DOD  to 
reduce  the  number  of  potential  weapon  system  mishaps 
resulting  from  software  errors.  All  of  these  tasks,  whan 
selectively  applied  and  properly  tailored,  provide  the  SSM 
with  an  effective  set  of  management  tools  to  implement  a 
system  safety  program. 

Software  Safety  Analysis  Techniques 

This  section  examines  six  structured  software  safety 
analysis  techniques  and  methodologies.  Three  of  the  six 
techniques,  Real  Time  Logic,  Petri  Nets,  and  Fault  Tree 
Analysis,  are  prominent  in  the  open  literature.  Three 
additional  techniques.  Fault  Hazard  Analysis,  software  snenk 
Circuit  Analysis,  and  Nuclear  Safety  Cross-Check  Analysis 
were  identified  during  a  survey  of  aerostpace  industry  and 
DOD  representatives. 

Many  other  forms  of  software  safety  analysin  were 
identified  in  the  literature  and  discussed  during  the 
survey.  A  list  of  these  other  techniques  is  included  in 
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Appendix  C.  Th«  following  ••otlons  d«scrib«  the  elx 
structured  techniques  and  methodologies  mentioned  above. 


Rfifll.  TimB  LQqlg>  Real  Time  Logic  (RTL)  provides  the 

basis  for  a  software  safety  analysis  teohnigue  which  uses  a 

formal  logic  "especially  amenable  to  reasoning  about  the 

timing  behavior  of  systems"  (23t890).  This  approach 

concentrates  on  the  system  specification  rather  than  the 

final  program  design  (23t891).  The  Real  Time  Logic  analyels 

process  begins  with  the  system  designer  choosing  "a  model  of 

the  system  In  terms  of  events  and  actions"  (24il42).  The 

purpose  of  the  "event-action  model  le  to  capture  the  data 

dependency  and  temporal  ordering  of  the  computational 

actions  that  must  be  taken  In  response  to  events  In  real 

time  applloatlon[s]"  (23S891).  The  event-action  model  Is 

then  "mechanically  translated  Into  Real  Time  Logic  formulas" 

(241142).  An  example  of  a  Real  Time  Logic  formula  Is  shown 

in  Figure  4  and  Interpreted  as  follows] 

Each  occurrence  of  the  event  denoting  the  completion  of 
action  A  happens  at  least  o(A)  time  units  after  the 
corresponding  occurrence  of  the  start  event,  where  o(A) 
denotos  the  specified  execution  time  of  action  A. 
(23i897} 


VtiVtaVi[«(tA,  1)  -  ti  A  S  e(iA,  1)  -  tj]-^ 

ti  +  0(A)  i  ta 

Figure  4.  Real  Time  Logic  Equation  (23:897) 
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To  oonploto  th«  analyala  of  tha  ayatant,  "tha  RTL 
formulaa  ara  tranafomad  Into  pradlctora  of  Praaburgar 
arithmatio  with  unlntarruptad  Intagar  funotions”  (24:143). 
Prom  thaaa  funotionii  «axiatlng  [Praaburgar  arithmatlc] 
prooaduraa  oan  ba  uiiad  to  datarmlna  If  a  glvan  aafaty 
aaaartlon  ia  a  thaoram  darlvabla  from  tha  ayatama 
apaolfioation"  (23:901).  If,  for  axampla,  tha  formula  in 
Figura  4  rapraaantod  a  aafaty  aaaartion  provan  by  RTL  to  bo 
trua,  than  "tha  ayatam  ia  aafa  with  raapaot  to  tha  timing 
bahavior  danotad  by  that  aaaartion  aa  long  aa  tha 
implamantation  aatiafiaa  tha  raqtulramanta  apaoifioation" 
(24:143).  Otharwiaa,  "tha  ayatam  ia  inharantly  unaafa” 
(6:143).  Raal  Tima  Logic  taohniquaa  ara  atill  baing 
davalopad.  Whila  Jahanlan  and  Mok  did  not  Inoluda  an 
explanation  of  Praaburgar  arithmatio,  thay  nota  that  tha 
mathamatioal  prooaaa  involving  full  implamantation  of 
Praaburgar  arithmatio  ia  axpanalva  and  inaffioiant  (23:900). 

Patri  Nata.  Patri  Nata  inoorporata  ayatam  timing 
raquiramanta  into  tha  analyaia  of  aoftwara  aafaty  (28:386). 
Patri  nata  "allow  mathamatioal  modeling  of  diaorata-avant 
ayatama  in  tarma  of  oonditiona  and  avanta  and  the 
ralationahip  between  them"  (24:143).  Unique  aymbola  ora 
uaad  to  rapraaant  plaoaa  (p) ,  tranaitiona  (t) ,  and  inputa 
(i)  and  outputa  (o)  on  a  Patri  net  graph.  Tha  input 
funotiona  ara  than  atudiad  to  datarmina  their  effect  on  tha 
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output  of  tho  sygtom  (28:387).  An  oxanplo  of  a  Patrl  net 
graph  for  a  railroad  oronslng  ie  shown  in  Figure  5. 


Figure  S.  Petri  Net  Graph  (28:387) 


Petri  net  analysis  is  similar  to  Real  Time  Logic  in 
that  both  rely  on  mathsmatloal  modeling  of  events  within  the 
system  (24:143).  Onue  the  Petri  net  models  are 
constructed,  however,  they  are  analyzed  through  simulation 
and  mathematical  techniques  "to  determine  desirable  and 
undesirable  properties  of  the  design"  focusing  on  the 
probability  of  simultaneous  events  (24:143).  The  inclusion 
of  simultaneous  events  is  an  important  consideration  in 
Petri  net  analysis  because  many  safety-critical  systems  fail 
not  on  the  basis  of  one  failure,  but  usually  os  a  result  of 
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a  series  of  failures  associated  with  hardware,  software, 
operator,  or  any  combination  of  the  three  (33:905). 

Another  unique  aspect  of  Petri  nets  Is  that  potential 
failures  can  be  modeled  and  their  effects  on  the  system 
studied  (28:392).  The  modeling  of  potential  failures  can  be 
very  useful  In  the  analysis  of  safety  critical  systems  where 
the  goal  is  to  reduce  the  probability  of  failure  by 
eliminating  unnecessary  events  within  the  system. 

Petri  nets  also  allow  the  use  of  backward  analysis 
procedures  "to  determine  which  failures  and  faults  are 
potentially  the  most  hazardous  and  therefore  which  parts  of 
the  system  need  to  be  augmented  with  fault-tolerance  and 
fail-safe  mechanisms"  (24:143).  Backward  analysis  Is  an 
important  factor  In  the  consideration  of  a  software  analysis 
technique  because  the  earlier  a  potential  failure  Is 
detected,  the  easier  It  Is  to  modify  the  software 
requirements. 

Although  there  are  advantages  associated  with  Petri 
nets  as  a  software  safety  analysis  tool,  the  development  of 
a  Petri  net  model  for  a  complete  system  Is  difficult  and 
time  consuming  (24:143).  Computational  time  and  computer 
memory  requirements  can  easily  make  analysis  of  a  complete 
system  Impractical  (42:E-25).  In  addition,  the  success  of 
the  final  analysis  depends  on  the  analyst's  knowledge  of  the 
system  and  familiarity  with  the  Petri  net  modeling  technique 
(42:E-25) . 
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Software  Fault  Tree 


Analyels,  also  know  as  Soft  Tree  Analysis,  is  a  software 
safety  analysis  technique  similar  to  Fetri  nets  in  that  it 
tries  to  "assess  safety  using  a  backward  approach  that 
starts  with  determining  what  are  the  unacceptable  or 
haaardous  states  and  then  showing  that  these  states  cannot 
occur  or  that  their  probability  of  ooourrenoe  is  low" 
(27:49).  Fault  Tree  Analysis  differs  from  Real  Time  Logic 
and  Petri  Nets  in  that  the  internal  timing  of  the  system  is 
not  considered  in  the  analysis. 

Fault  trees  "depict  the  logical  interrelationships  of 
basic  events  that  lead  to  a  system  hazard"  (4:378). 

Figure  6  shows  an  example  of  a  complete  fault  tree  that  v;as 
used  to  analyze  software  written  to  control  a  traffic  light 
(4:383) . 

The  root  of  the  fault  tree  in  Figure  6  is  the  hazard; 
the  analysis  determines  whether  the  hazard  can  occur  due  to 
a  software  failure.  The  lower  levels  of  the  tree  are  known 
as  preconditions  and  are  "described  at  the  next  level  of  the 
tree  with  either  a  logical  AND  or  a  logical  OR  relationship" 
(27:49).  Once  the  fault  tree  has  been  constructed,  the 
analysis  can  begin  to  determine  whether  the  right 
combination  of  preconditions  can  be  met  leading  to  the 
hazard. 


45 


Figure  6.  Software  Fault  Tree  (4:383) 
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The  developare  of  Real  Time  Logic  have  oritiolzed  fault 
tree  analysie  stating  that  It  relies  "heavily  on  human 
Inspection  and  that  fault  tree  analysis  makes  no  attempt  to 
formally  capture  the  timing  behavior  of  a  system"  (23:891). 
Although  fault  tree  analysis  rellei  heavily  bn  human 
Inspectlbn,  the  dependence  of  software  safety  upon  analysis 
of  timing  constraints  has  yet  to  be  proven.  Software  fault 
tree  analysis  does  "force  the  programmer  or  analyst  to 
consider  what  the  software  Is  not  supposed  to  do"  (4:377). 
Although  the  development  of  fault  tree  analysis  began  nearly 
30  years  ago  for  the  purpose  of  analyzing  the  safety  of 
eleotromeohanloal  systems,  the  teohnlgur  as  It  applies  to 
software  Is  still  being  perfected  (27:49).  Software  Fault 
Tree  Analysis  can  be  difficult  to  apply  to  a  complex  system 
and  the  reliability  and  success  are  dependent  upon  the 
experience  of  the  analyst  (4:386). 

,  ftnalYiili .  Another  software 

safety  analysis  technique  adapted  from  hardware  reliability 
and  safety  analysis  Is  known  as  Software  Sneak  Circuit 
Analysis  (SSCA) .  A  report  written  by  the  Boeing  Aerospace 
Company  for  Rome  Air  Development  Center,  Or Iff las  AFB,  NY, 
defines  Sneak  Analysis  as 

. . H  an  engineering  analysis  tool  which  can  be  used  for 
hardware  and  software  systems  to  Identify  latent  paths 
which  cause  concurrent  unwanted  functions  or  Inhibit 
desired  functions,  assuming  all  component  and  codes  are 
functioning  properly.  (3:1) 
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The  primary  objective  of  SSCA  le  to  Identify  potential 

safety  problems  within  a  system's  software  so  that  they  can 

be  assessed  and,  If  warranted,  corrected  before  the  systein 

begins  testing  or  Is  plaoed  Into  operation  (3tl). 

The  first  step  In  the  SSCA  teohnlgue  is  to  convert  the 

program  source  code  to  a  network  tree  representation,  "such 

that  program  control  Is  considered  to  flow  down  the  page" 

(3i66)«  Difficult  programs  may  bo  converted  by  computer, 

whereas  simpler  programs,  based  on  the  experience  of  the 

analyst,  can  be  converted  manually  (3s 66). 

Once  the  topological  network  trees  have  been  drawn,  the 

next  step  Is  to  Identify  "the  basic  topological  patterns 

that  appear  In  the  trees"  (3s68).  There  are  six  basic 

patterns  shown  In  Figure  7.  Any  software  program  can  be 

represented  by  a  combination  of  these  basic  topological 

patterns  (3i68)«  According  to  the  Boeing  Aerospace  Reports 

The  topological  patterns  containing  branch  or  jump 
Instructions  have  the  highest  Incidence  of  problems. 
This  Includes  the  Return  [Dome],  iteration  [/Loop 
Circuit]  and  Parallel  Line  ....  (3:68) 

These  patterns  result  In  the  most  errors  primarily  because 

of  the  difficulty  Involved  In  crossing  module  or  function 

Interfaces  within  the  code  (3s 68).  Common  errors  are 

unintentional  modification  of  memory  contents,  attempts  to 

use  uninitialised  variables,  and  timing  or  sequencing 

problems  (3s 68). 
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Flgur*  7.  Software  Topographs  (3t68) 


Tht  nsKt  stop  In  an  SSCA  Is  to  analyst  ths  baslo 
struoturss  within  tht  topographic  trtt.  Thtst  struoturas 
art  analyrtd,  bastd  on  thtir  historical  oharaottristlos,  to 
dtttmint  whtthtr  any  sntak  conditions  txlst.  Four  baslo 
typts  of  software  sntak  conditions  txlst > 

1.  Sneak  Output  -  the  occurrence  of  an  undtsirtd 
output . 

a.  Sntak  inhibit  -  the  undtsirtd  Inhibition  of  an 
input  or  output. 

3.  Sntak  Timing  -  the  occurrence  of  an  undasirtd 
output  by  virtue  of  its  timing  or  mismatched  input 
timing. 

4.  Sneak  Message  -  the  problem  message  does  not 
adequately  reflect  the  condition  (3t68). 
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An  Identified  software  eneak  must  be  verified  through  a 
complete  analyeie  of  the  source  code.  If  the  software  sneak 
is  proven  to  be  valid,  "a  Software  Sneak  Report  ...  is 
written  which  includes  an  explanation,  system  level  impact, 
and  a  recommendation  for  elimination  of  the  sneak"  (3i69). 

Software  Sneak  Circuit  Analysis  is  a  technique  that  may 
bo  used  "to  identify  system  conditions  that  could  degrade  or 
adversely  impact  the  mission  safety  or  basic  equipment 
reliability"  (3t74).  According  to  the  Boeing  Aerospace 
Company  report,  SSCA  "should  be  considered  for  critical 
systems  and  functions  where  other  techniques  are  not 
effective  .  .  ."  (3i74). 

Although  the  cost  of  performing  Sneak  Analysis  is  high 
for  both  hardware  and  software,  it  is  important  to  note  that 
"regardless  of  the  program  development  phase,  application 
environment,  equipment/software  type,  criticality  ranking, 
or  program  cost.  Sneak  Analysis  identifies  a  significant 
number  of  system  problems"  (3tl41). 

ZttUlt,  Hflgard  Analysis*  Fault  Haisard  Analysis  (FHA)  is 
an  inductive  analysis  methodology  which  has  its  origins  in 
hardware  analysis  but  which  can  be  tailored  to  software 
analysis.  As  such,  FHA  is  sometimes  referred  to  as  software 
Fault  Hazard  Analysis  (SFHA)  (19i21).  FHA  is  used  primarily 
to  generate  a  qualitative  analysis  but  can  be  extended  to 
include  some  quantitative  data.  The  FHA  methodology 
involves  the  application  of  engineering  principles  and 
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Induotlv*  roasonlng  to  d«t«nnin«  "what  can  fall,  how  it  can 
fall,  how  frequently  It  will  fall,  what  the  effeots  of  the 
failure  are,  and  how  Important  the  effeote  of  the  failure 
are"  (10:28). 

FHA  le  elmllar  In  form  to  the  Failure  Node,  Effects, 
and  criticality  Analysis  (FMBCA)  of  MIL-STD-1629A.  The 
FNBCA  Is  a  reliability  analysis  which  looks  at  all  possible 
single  component  failures  in  a  system.  The  FHA,  on  the 
other  hand,  must  focus  cnly  on  safety  related  failures  and 
effects.  The  FMBCA  also  has  a  software  derivative  called 
floftware  Failure  Modes  and  Effects  Analysis  (SFMBA) 

(42IE-30).  The  FMBCA  cften  serves  as  a  starting  point  for 
an  FHA. 

Fault  Hasard  Analysis  appears  to  have  Its  best 
applicability  early  In  the  design  phases  of  a  system  as  It 
requires  knowledge  of  requirements  allocation  to  start  the 
effort.  As  the  design  progresses,  the  FHA  becomes  more 
detailed.  The  FHA  should  prepare  data  for  use  by  subsequent 
deductive  analysis  techniques  (11:42). 

FHA  Is  limited  In  that  it  treats  software  Independent 
of  hardware  and  can  be  very  lengthy  and  tedious.  The 
fundamental  Ingredient  for  making  the  FHA  specifically  a 
software  safety  analysis  technique  Is  Its  execution  by  a 
software  specialist.  An  accurate  and  complete  analysis 
requires  an  experienced  analyst  (42:B->30). 
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MagliaE-aftgltY..CrQiirfihtgh  .AnolVlii  ■  a  mathodology 

dttvelopad  to  satisfy  Air  Foro*  rsqulrsinsnts  for  nuclear 

safety  is  the  Nuclear  Safety  Cross-Check  Analysis  (NSCCA) . 

NSCCA  takes  an  adversarial  approach  to  software  analysis  by 

employing  a  large  number  of  techniques  to  '‘show,  with  a  high 

degree  of  confidence,  that  the  software  will  not  oontribute 

to  a  nuolaar  mishap"  (24 t 145).  The  NSCCA  process  has  both  a 

teohnioal  and  prooedural  oomponent.  The  objective  of  the 

teohnioal  component  is  to  evaluate  the  software  by  test  and 

analysis  to  onsure  safety  requiremente  are  met.  The 

procedural  oomponent  focuses  on  implementing  security  and 

control  measures  to  "protect  against  sabotage,  collusion, 

compromise,  or  alteration  of  critical  software  components, 

tools,  and  NSCCA  results"  (isiis). 

The  teohnioal  oomponent  of  NSCCA  starts  with  two 

distinct  steps  which  comprise  a  criticality  analysis 

(13115).  The  first  step  identifies 

. . .  specific  requirements  that  are  the  minimum  positive 
measures  necessary  to  demonstrate  that  the  . . .  software 
is  predictably  safe  according  to  general  DOD  standards 
for  nuclear  systems.  (24 i 146) 

The  second  step  Involves  breaking  down  the  software  system 
into  its  lowest  order  functions  and  analysing  each  function 
for  the  degree  to  which  it  controls  or  operates  on  a 
critical  nuclear  event  (I3tl5).  The  degree  each  function 
influences  a  critical  nuclear  event  is  given  a  qualitative 
(high,  medium,  low)  rating,  and  the  best  method  to  measure 
each  function  is  suggested. 
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A  program  managar  uaaa  tha  reaulta  of  the  orltloallty 
analyeia  to  develop  an  NSCCA  program  plan  whloh  establishes 
the  required  tools  and  facilities;  required  analysis;  and 
test  requirements,  planning,  and  p.rcoedures  (13;  15).  The 
independence  of  an  NSCCA  is  promoted  by  early  establishment 
of  evaluation  criteria,  purpose,  objectives,  and  expecbed 
results  for  speoifio  analyses  and  tests  (13; 15). 

The  procedural  component  of  NSCCA  has  as  Its  objective 
the  proteotion  of  both  the  software  and  analysis  environment 
from  any  possible  oompromlse  or  alteration  (13; 16).  This 
objective  is  met  by  "instituting  strong  personnel,  dooumant, 
and  facility  security  measures,  coupled  with  stringent 
product  control  procedures"  (13; 16), 

NSCCA  has  the  advantage  that  it  ic  usually  aooomplishad 
by  an  agency  or  contractor  who  is  independent  of  the 
software  developer  (13;l4).  In  addition,  it  is  applicable 
at  any  stage  of  the  software  development  cycle  (24; 146). 

The  limitation  of  NSCCA  is  that  its  effectiveness  and 
expense  depend  entirely  upon  the  specific  analyses  and  test 
procedures  selected  (42;B*‘a4). 

Software  Safety  Analysis.  ■Taohnicue ■Summary.  The  six 
software  safety  analysis  techniques  and  methodologies 
disousssd  above  provide  a  good  representation  of  the  types 
of  software  safety  analyses  currently  available.  All  are 
considered  structured  in  that  they  can  be  applied  with  some 
rigor  to  produce  consistent  results. 
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All  of  thtt  t«chniqu«B  ar*  •xpanalva  and  tltna  oonauMlng 
when  appllad  to  oonplax  aafaty-orltloal  ByatcmB.  Each 
reliBB  on  th*  analyst 'b  aapabillty  and  familiarity  with  the 
applloablB  analyalB  tBohnlquB  or  aothodology. 

Although  Baoh  tBohnlgu#  can  provide  UBtful  information 
at  different  phaeea  of  the  development  prooeea^  none  oan  yet 
guarantee  an  absolutely  safe  syetem.  Reeearoh  ie  on-golng 
to  improve  the  effectiveneea  of  each  technique  and  to 
automate  the  safety  analysis  prooess  as  much  as  poBslble  to 
eliminate  analyst  depandenoy. 


This  chapter  explored  the  relationships  between  the 
software  development  prooess,  software  and  system  safety 
program  requirements,  and  software  safety  analysis 
teohniques.  Figure  8  shows  the  relationships  between  these 
three  components  of  a  software  safety  program.  This 
information  provides  an  idealised  understanding  of  where 
system  and  software  safety  requirements  ore  addressed  within 
the  software  development  prooess,  what  the  system  and 
software  safety  program  requirements  are,  and  how  the 
software  safety  ana].ysis  requirements  are  met. 

Software  safety  should  and  oan  be  considered  throughout 
the  software  development  process.  The  software  development 
process  of  D0D»STD’-2167A  includes  numerous  reviews  and 
audits  that  provide  ample  opportunity  to  review  software  and 
system  safety.  The  reviews  and  audits,  as  defined  by 


Figure  8.  Software  Safety  Frogran  Component  Relationehipe 


MIL-STD-ISSIB,  each  contain  epeciflc  software  end  eyatem 
safety  agenda  Items  for  evaluation  or  review. 

Specific  safety  program,  hazard  analysis,  and  risk 
assessment  requirements  are  provided  by  NIL’-STD~882B.  The 
requirements  tell  what  analyses  and  assessments  should  be 
performed  at  different  phases  of  a  development  process.  In 
addition,  the  requirements  tell  the  type  and  reporting 
format  of  the  analysis  data.  NIL->STD-882B  does  not, 
however,  specify  how  the  analysis  data  are  to  be  obtained. 
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Hazard  analyala  and  riak  aaaasamant  data  oan  ba 
produced  by  structured  software  safety  analysis  techniques. 
There  are  many  analysis  techniques  that  can  aid  a  system  or 
software  safety  analysis;  however,  we  encountered  only  six 
structured  analysis  techniques  have  been  developed  or 
adapted  specifically  for  software  safety  analysis.  While 
these  structured  techniques  can  provide  the  required 
software  safety  data,  they  are  currently  limited  In 
application  as  well  as  difficult  and  expensive  to  apply. 
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Introduction 

This  chapter  analyzes  the  results  of  a  survey  of 
aerospace  Industry  and  DOD  personnel  conducted  to  determine 
what  software  safety  analysis  techniques  are  actually  used 
in  DOD  weapon  systems  development  and  what  successes  or 
failures  have  been  realized  by  their  application.  The  four 
sections  of  this  chapter  discuss  the  characterization  of  the 
sample  population,  survey  questions,  survey  responses,  and 
conclusions. 


The  survey,  located  In  Appendix  D,  was  administered  to 
a  sample  population  that  consisted  of 

«  Recognized  experts  in  software  safety  analysis 

•  System  and  software  safety  engineers  and  managers 

*  Project  engineers  and  managers 

The  Individuals  selected  for  this  survey  represent  a 
cross-section  of  professionals  throughout  the  aerospace 
Industry  and  DOD  that  are  experienced  In  the  system  or 
software  safety  disciplines. 

There  were  54  Individuals  contacted  throughout  the 
survey  process.  However,  only  23  were  experienced  in 
software  development  and/or  software  safety.  These  23 
individuals  were  selected  to  participate  in  this  study  and 
became  the  sample  population. 
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The  target  organizations  for  this  study  consisted  of 
aerospace  Industry  ^nd  DOD  representatives  Involved  In 
system  ^nd  software  safety.  Table  7  shows  the  break-out,  by 
organlii.atloi(,  of  participants  between  the  aerospace  industry 


Table  7.  Survey  Participant  Organizations 


Aarospaca 

Industry 

DOD 

Other 

Total 

Military 

Civilian 

Total 

Tartlclpants 

8 

4 

6 

5 

23 

Parcant 

34.8 

17.4 

26.1 

21.7 

100.0 

43.5 

and  DOD.  The  aerospace  Industry  participants  account  for 
34.8  percent  of  the  sample  population  and  the  combi: ;.rrlon  of 
DOD  military  and  civilian  participants  account  for  43.5 
percent  of  the  sample  population. 

The  "Other"  classification  In  Table  7  consists  of 
survey  participants  that  did  not  fit  Into  the  Aerospace 
Industry  or  DOD  classification.  They  Include  three  DOD 
support  contractors,  one  software  safety  consultant,  and  one 
research  Institute  representative.  As  shown  In  Table  7,  the 
"Other"  category  accounts  for  21. V  percent  of  the  sample 
population. 

The  target  professions  for  this  study  consisted  of 
system,  software,  and  project  engineers  and  managers. 

Table  8  shows  the  prr'fesslonal  classification  of  the  survey 
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Table  8.  Survey  Participant  Professions 


System/Softwara 

Pro' 

act 

Other 

Total 

Engineer 

Manager 

Engineer 

Manager 

Total 

6 

6 

3 

5 

3 

23 

Fercant 

26.1 

26.1 

13.0 

21.8 

13.0 

100.0 

52. 2 

participants.  The  System/Software  category  aooounts  for  the 
majority  of  the  participants  at  5:^.2  percent.  V7e  consider 
this  to  be  an  advantage  because  these  Individuals  were  most 
knowledgeable  about  the  mechanics  of  applying  software 
safety  analysis  techniques.  The  34.8  percent  In  the  project 
category  provided  the  most  Information  concerning  the 
management  aspects  of  software  safety. 

The  13.0  percent  In  the  "Other"  classification  of 
Table  8  provided  Infoxmatlon  concerning  both  the  application 
and  management  aspects  of  software  safety  analysis 
technique.').  They  included  one  software  safety  consultant 
and  two  research ^ars. 

Based  on  the  above  date  we  believe  that  each  of  the 
targeted  organizations  and  professions  Is  well  represented 
in  this  study.  Therefore,  the  survey  results  provide  an 
accurate  picture  of  the  currant  state  of  software  safety  as 
It  relates  to  bOC  weapon  system  development.  Additional 
Infottnatlon  on  each  member  of  the  sample  population  is 
included  In  Appendix  E. 
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The  survey  questions  were  structured  as  open-ended 
questions  to  gain  a  first-hand  account  of  how  an 
organization  manages  and  conducts  system  and  software  safety 
programs.  Typically,  the  survey  question  was  just  a  lead-in 
to  a  more  detailed  discussion  of  software  safety.  Each 
question  from  the  survey  Is  presented  below  followed  by  Its 
Intended  purpose. 

Question  1.  Do  you  work  with  the  DOD  software 
development  process  as  defined  In  DOD-STD-2167A  or  some 
close  derivative? 

This  question  established  survey  participants' 
familiarity  with  the  DOD's  software  development  process. 

The  question  served  as  the  first  of  two  filters  to  narrow 
the  pool  of  potential  participants. 

Question  2.  Is  software  safety  or  system  safety  an 
Issue  that  is  faced  In  your  organization? 

This  question  established  survey  participants' 
familiarity  with  software  and/or  system  safety.  The 
question  served  as  the  second  filter  to  narrow  the  pool  of 
participants  down  to  the  sample  population. 

Queetlon  3.  Is  software  safety  considered  internal  or 
external  to  the  overall  software  development  process? 

This  (question  assessed  the  relative  Importance  of 
software  safety  as  part  of  an  organization's  overall  system 
or  software  development  process.  For  this  question,  we 
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oonslderad  an  Intarnal  softwara  aafaty  program  as  ono  that 
was  fully  Integratad  into  the  software  development  prooess 
and  addressed  safety  Issues  throughout  the  process  phases. 

An  external  software  safety  program,  on  the  other  hand,  was 
a  program  not  fully  Integrated  into  the  software  development 
process  and  only  addressing  Safety  Issues  at  a  few  discrete 
points  In  the  development  prooess. 

Question  4.  Are  there  any  specific  software  safety 
analysis  teohnlques  and/or  management  processes  related  to 
software  safety  employed  in  your  organisation?  Explain. 

This  question  identified  the  software  safety  techniques 
and  methodologies  used  by  survey  participants  to  address 
software  safety.  This  information  contributed  to  our 
assessment  of  the  degree  of  use  and  viability  of  structured 
software  safety  analysis  techniques. 

Question  5 .  What  successes  and/or  failures  has  your 
organiTiation  had  with  software  safety  analysis  techniques? 

This  question  established  the  viability  of  structured 
software  safety  analysis  teohnlques.  The  focus  was  on 
determining  what  criteria  survey  participants  considered  to 
be  good  measures  of  sucoess  and  failure  for  the  application 
of  these  techniques. 

Question  6.  In  your  opinion,  what  are  the  benefits  or 
drawbacks  to  applying  software  safety  analysis  techniques? 

This  question  helped  identify  strengths  and  weaknesses 
of  the  structured  analysis  techniques  as  well  as  management 
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and  training  Implloatlona  of  ualng  tha  taohnlquas. 

Attrlbutaa  of  spaolflc  taohnlquas  aa  well  as  structured 
analysis  techniques  as  a  group  were  sought. 

gucy.iY  Rtiponni 

synthesized  responses  to  the  survey  questions  are 
presented  below. 

Question  1 .  Do  you  work  with  the  DOD  software 
development  process  as  defined  in  DOD-STD-2167A  or  some 
close  derivative? 

All  survey  participants  working  on  DOD  programs 
Involving  software  development  follow  the  software 
development  process  defined  in  DOD-STD-SldTA  to  some  degree. 
As  expected/  the  development  process  phases  for  each  project 
discussed  during  the  survey  were  tailored  to  fit  unique 
project  situations  and  system  requirements.  Along  with 
MIL~STD-2167A/  all  projects  employ  some  form  of  a  quality 
program  plan  as  outlined  in  DOD-STD-2168/  Defense  System 
Sgftvarg-,. Quality. JPiQgram  ■  some  of  the  older  contracts 
discussed  during  Interviews  use  tailored  versions  of 
DOD-STD-2167  along  with  MIL-S-S2779A/  System  Software 
Quality  Program  (the  predecessor  to  DOD-STD-2168) .  Most  of 
these  older  contracts,  however,  are  close  to  completion  or 
in  the  process  of  being  updated  to  Include  the  latest 
standards.  Tha  more  recently  awarded  contracts  use 
DOD-STD-2167A  in  conjunction  with  DOD-STD-2168. 
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Quagtlon  2.  Is  softwars  safaty  or  ayatain  safety  an 
Issue  that  Is  faced  in  your  organization? 

All  survey  participants  Involved  In  DOD  weapon  system 
development  agreed  that  system  safety  Is  a  concern  that  they 
must  address  in  their  development  process,  in  addition, 
they  recognized  MIL-STD-882B  as  the  principal  guidance 
document  for  DOD  system  safety  program  requirements. 

Finally,  all  participants  recognized  that  software  Is  an 
Important  aspect  of  system  safety  and  knew  MIL-STD-882B 
contained  software  safety  analysis  tasks. 

Question  3.  Is  software  safety  considered  Internal  or 
external  to  the  overall  software  development  process? 

Although  all  survey  participants  generally  agreed  on 
the  Importance  of  software  safety  In  a  system  context,  it 
was  apparent  that  many  organizations  do  not  treat  software 
safety  as  an  Internal  part  of  their  development  process. 

Many  participants  Indicated  that  they  depended  on  a  thorough 
Failure  Modes  and  Effects  Analysis  and  overall  system  safety 
analyses  to  be  performed  by  their  reliability,  safety,  and 
systems  engineers.  The  software  requirements  along  with 
relevant  failure  mode  Information  are  then  handed  to  the 
software  developers  who  design,  code,  and  test  the  software 
products.  If  the  resultant  software  meets  design 
requirements  both  for  what  It  Is  Intended  to  do  and  for  what 
It  Is  not  Intended  to  do,  then  the  software  is  considered 
safe.  A  few  participants  stated  that  If  the  overall  system 
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■afttty  Is  addrssssd  throughout  davslopmsnt,  software  safety 
could  easily  be  verified  during  system  testing. 

Question  4.  Are  there  any  speolflo  software  safety 
analysis  techniques  and/or  management  processes  related  to 
software  safety  employed  In  your  organisation?  Explain. 

Most  survey  participants  recognised  Software  Fault  Tree 
Analysis  and  software  Petri  Net  Analysis  as  structured 
software  safety  analysis  techniques.  None  of  the 
participants,  however,  had  used  either  of  the  teohnlqiies  In 
performing  software  safety  analyses.  Many  survey 
participants  were  unfamiliar  with  Real  Time  Logic  and  none 
had  ever  used  the  technique. 

Survey  participants  Identified  Software  Sneak  circuit 
Analysis,  Nuclear  Safety  Cross-Check  Analysis,  and  Fault 
Hazard  Analysis  as  additional  structured  safety  analysis 
techniques  and  methodologies  applicable  to  software  safety. 
Of  these  later  techniques,  only  Fault  Hazard  Analysis  (FHA) 
was  widely  used.  In  addition,  many  participants  indicated 
they  use  various  general  purpose  analysis  and  Inspection 
tools  (Included  In  Appendix  C)  to  aid  In  their  analysis  of 
software. 

Two  Industry  survey  participants  perform  hardware  baaed 
fault  tree  analysis  as  part  of  their  overall  system  safety 
program.  This  analysis  Is  intended  to  analyze  hardware 
modules,  but  can  Include  software  by  default  If  the  software 
Is  embedded  In  the  hardware  module.  Another  Industry 
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participant  usaa  hardwara  basad  anaak  circuit  analyala  In 
much  tha  aama  way«  Nalthar  nathod,  howcvari  la  spaclfloally 
Intandad  for  aoftwara  aafety  analyala. 

Tha  pradoinlnant  method  for  performing  aoftwara  safety 
analyaaa  appaara  to  be  Fault  Hazard  Analyala  (FHA) .  FHA  la 
uaad  to  Identify  potential  hazarda  and  aafaty-orltloal 
aoftwara  In  a  ayatam.  Evan  thoaa  partlolponta  who  do  not 
haVa  a  apaqlflo  aoftwara  aafety  program  alao  employ  a  form 
of  FHA  to  Identify  aoftwara  related  ayatam  hazarda  and  than 
provide  a  Hat  of  thoaa  hazarda,  along  with  a  aat  of 
software  raqulramanta,  to  their  aoftwara  davalopars. 

purvey  participants  using  FHA  ware  ackad  If  there  are 
any  apaolflo  guidance  and/or  prooaduraa  for  applying  FHA 
methodology.  Raaponaaa  varied  for  each  participant, 
however,  a  general  pattern  waa  evident.  Steps  common  to  the 
FHA  methodology  Include  brainstorming  potential  hazards. 
Hating  known  hazarda  for  similar  ayatema,  and  performing 
probability  analyaaa  on  potential  hazarda.  No  Industry 
standard  waa  Identified  for  conducting  FHAs,  however,  each 
participant's  organization  had  some  general  guidelines.  As 
for  guidance,  all  partlolponta  pointed  to  MIL*‘STO~882B, 
System  Safety  Program  Reaulremtnta.  which  contains  tasks 
specifically  called  out  In  DOD  oontraota.  However,  survey 
participants  pointed  out  that  MIL-8TD»882B  only  provides 
details  of  what  specific  hazard  analyses  should  address,  It 
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do«B  not  provldB  guidanoB  a«  to  how  an  analyala  should  be 
oonductsd. 

Thsre  wars  no  speolfic  nanagsment  procssses  Identified 
to  insure  software  safety  is  included  in  product  design  and 
development.  There  was  however,  a  sense  of  management 
oommitnent  toward  th«  development  of  a  safe  system  which 
includes  safe  software.  Each  project  encountered  has,  on 
both  the  oontraotor  and  government  teams,  someone 
responsible  for  system  safety.  Contractors  typically  have  a 
number  of  safety  and  system  engineers  involved  in  the  system 
design  ^nd  development  while  the  government  typically  has  a 
single  System  safety  Manager  (SSM) , 

A  follow-up  guestion  was  asked  of  survey  participants 
concerning  the  qualifications  and  training  of  safety 
personnel.  Responses  indicated  that  both  industry  and  DOD 
personnel  are  often  assigned  system  safety  duties  with 
minimal  safety  qualifications.  Some  of  the  participant's 
organizations  have  developed  their  own  system  safety 
training  courses.  These  purely  introductory  level  courses 
are  from  1  to  2  weeks  long  and  are  aimed  at  project 
managers,  SSMs,  safety  engineers,  and  system  engineers, 
other  industry  organizations  require  their  safety  personnel 
to  obtain  system  safety  oertlfloation  through  college  or 
technical  school  courses.  While  all  of  these  courses 
Include  some  software  safety  discussion,  none  are  devoted 
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solely  to  Boi'twars  safety  or  software  safety  analysis 
techniques. 

A  second  follow-up  question  was  asked  of  participants 
concerning  their  perceived  need  to  acquire  analysis 
capabilities  with  one  or  more  of  the  structured  software, 
safety  analysis  techniques  described  in  Chapter  III.  Most 
participants  were  quick  to  point  out  that  they  were  fully 
capable  of  meeting  current  contractual  requirements  With  the 
FHA  methodology.  The  general  consensus  among  contractors 
was  why  spend  time  and  money  on  obtaining  structured 
software  safety  analysis  capability  that  the  government  has 
not  asked  fori 

Question  B.  What  successes  and/or  failures  has  your 
organization  had  with  software  safety  analysis  techniques? 

As  none  of  the  structured  techniques  were  employed  by 
survey  respondents,  there  were  no  documented  sucoessea  or 
failures  to  be  discussed.  Even  though  FHA  was  the 
predominant  method  used  to  ensure  software  safety,  it  is 
viewed  by  participants  as  an  inductive  methodology  that  may 
include  a  number  of  structured  techniques.  Survey 
participants  found  it  impossible  to  quantify  how  successful 
FHA  has  been.  It  was  pointed  out  that  simply  having  a 
methodology  to  use  is  better  than  no  analysis  at  all.  None 
of  the  participants  were  able  to  define  a  measure  for  failed 
application  of  a  technique. 


67 


Quaatlon  6.  In  your  opinion,  what  art  tho  banaflts  or 
drawbacks  to  applying  softwara  safety  analysis  teohnlq^ies? 

All  participants  agreed  that  the  biggest  benefit  to  be 
realised  in  the  ouddessful  application  of  software  safety 
analysis  is  a  higher  quality  product.  They  also  agreed  that 
an  increase  in  quality  will  lead  to  increased  functionality, 
reliability,  and  aalnta inability. 

There  were  a  number  of  disadvantages  discussed 
concerning  the  application  of  currently  known  structured 
software  safety  analysis  techniques.  The  most  common 
response  indicated  that  the  techniques  as  a  group  are 
difficult  to  learn  and  expensive  to  put  into  practice. 
Furthermore,  the  accuracy  of  an  analysis  depends  almost 
entirely  on  the  analyst's  skill  with  the  technique  and 
knowledge  of  the  system  being  analysed.  Finally,  it  was 
pointed  out  by  a  number  of  participants  that  these 
techniques  need  further  refinement  to  bring  them  from  a 
purely  theoretical  realm  into  a  more  practical  application 
realm. 

Follow-up  discussions  were  held  with  a  number  of  DOD 
survey  participants  concerning  the  adequacy  of  contractor 
delivered  safety  analyses.  They  indicated  that  there  appear 
to  be  two  principal  problems.  First,  program  offices 
typically  task  contractors  to  perform  only  system  level 
safety  analyses  (200  series  tasks  in  NIL-STD-682B)  that 
treat  software  as  a  secondary  issue.  It  was  pointed  out 
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that  avan  though  MIL»STD-882B  doaa  contain  savan  300  aarlaa 
taaka  apaolfloally  davotad  to  aoftwara  aafaty  analysis, 
program  offioas  opt  for  tha  200  sarias  analysas  because  they 
are  cheaper  to  obtain  and  easier  to  understand.  Second > 
there  appears  to  be  a  oonoarn  that  avan  if  tha  program 
offioas  are  willing  to  require  tha  300  sarias  analysas, 
there  are  few  individuals  sufficiently  trained  to  understand 
or  evaluate  tha  resultant  software  safety  analysis  reports. 

s.ugYiy  jeonglutioni 

There  is  a  general  awarenena  within  the  aerospace 
industry  and  OOD  of  the  need  for  software  safety.  While 
survey  participants  understood  that  software  plays  a  key 
role  in  system  safety,  it  was  disappointing  to  observe  that 
many  organizations  do  not  have  an  integrated  software  safety 
program.  There  is  also  no  standard  management  methodology 
being  followed  to  ensure  software  safety  is  adequately 
incorporated  into  weapon  system  development. 

The  aerospace  industry  and  DOD  do  not  currently  have 
any  standard  structured  technique  to  perform  software  safety 
analyses.  The  Fault  Hazard  Analysis  methodology  is  moat 
often  used,  but  specific  steps  in  tha  methodology  vary  from 
one  organization  to  the  next.  In  addition,  the  aerospace 
industry  typically  responds  to  what  the  customer  (DOD)  is 
asking  for,  and  in  DOD  weapon  system  development,  the 
customer  is  not  asking  for  structured  software  safety 
analysis  techniques  to  be  used.  Most  software  safety 
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analysas  ara  parformad  aa  part  of  a  ayatam  aafoty  analyais 
whara  aoftwara  la  oftan  not  adaquataly  addraaaad. 

Tha  primary  banaflta  of  atruoturad  aoftwara  safety 
analyala  ara  Inoraaaad  ayatam  aafaty  and  (juaXlty.  For  tha 
DOD  to  raallca  thaaa  banaflta,  aoftwara  davalopara  muat  ba 
required  to  uaa  atruoturad  aoftwara  aafaty  analyala 
tiohnlquaa.  All  of  tha  Induatry  partlolpanta  Indicated  they 
could  develop  tha  capability  to  apply  any  of  tha  atruoturad 
analyala  taohnlquaa  If  there  waa  a  apaolfio  demand  for  them. 
Tha  DOD,  however,  la  not  aakinq  for  tha  detailed  aoftwara 
aafaty  analyaaa  that  would  require  the  uaa  of  atruoturad 
taohnlquaa. 

Evan  if  tha  DOO  did  raquaat  tha  detailed  aoftwara 
safety  analyaaa,  it  is  unlikely  the  typical  DOD  syatem 
safaty  Manager  could  understand  and  evaluate  the  resultant 
products,  Tha  first  step,  therefore,  to  realising  tha 
benefits  of  atruoturad  aoftwara  safaty  analysis  taohnlquaa 
la  to  make  SSMa  aware  of  thaaa  taohnlquas,  thalr  benefits 
and  limitations,  and  provide  them  with  tha  training 
naoesaary  to  affaotlvaly  Implement  and  manage  a  software 
safaty  program. 
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Systtn  Safety  Managers  throughout  the  DOD  are 
reeponalble  for  Inplenenting  and  managing  eyatem  safety 
programs.  As  weapon  systems  become  more  software  dependent^ 
their  oorrespondlng  system  safety  programs  must  Include  a 
larger  focus  on  the  software  that  will  control  the  system. 
Interviews  with  aerospace  Industry  safety  managers 
substantiate  the  need  for  Increased  emphasis  on  software 
safety.  To  effectively  Implement  and  manage  a  system  safety 
program,  DOD  system  Safety  Managers  (SSM)  must  be  trained  In 
the  software  acquisition  process  and  software  safety 
procedures.  Interviews  with  ssMs  revealed  that  most 
consider  themselves  Inadequately  trained  to  effectively 
Implement  and  manage  the  software  portion  of  a  system  safety 
program. 

This  analysis  explores  SSM  training  and  education 
requirements  within  the  DOD  by  studying  ssMs  located  at 
Aeronautical  Systems  Center,  Wrlght-Patterson  AFB,  Ohio, 
Aeronautical  Systems  Center  (ASC)  SSMs  ere  fully  integrated 
Into  their  system  program  offices  and  represent  over  43 
percent  of  the  Air  Force  SSM  corps.  The  methodology  of  this 
analysis  consists  of  reviewing  current  ASC  system  safety 
training  materials,  system  safety  guidance  and  policy 
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doou?nentS|  and  system  and  subsystem  hazard  analysis 
documents.  In  addition,  SSMs  from  each  participating 
program  office  were  Interviewed  for  their  perceptions  of 
necessary  prerequisites  to  becoming  fully  qualified  SSMs. 

The  following  ASC  systom  program  offices  participated 
In  this  study: 

•  C-17  System  Program  Office 

•  Electronic  Combat  and  Reconnaissance  (RW)  System 
Program  Office 

•  F~16  System  Program  Office 

•  F>22  System  Program  Office 

These  programs  are  a  representative  sample  of  the  types  of 
programs  SSMs  may  encounter.  The  four  system  program 
offices  also  account  for  eight  SSMs,  which  comprise  over  25 
percent  of  the  SSM  force  at  ASC.  A  complete  list  of  the 
documents  reviewed  from  each  system  program  office  Is 
included  In  Appendix  F  and  a  list  of  the  corresponding  SSMs 
interviewed  is  provided  in  Appendix  E. 

This  analysis  begins  with  a  verification  of  the  need 
for  software  specific  training  at  ASC  by  providing  a 
snapshot  of  the  current  hardware  and  software  qualifications 
of  SSMs.  Noxt,  this  analysis  describes  the  SSM's  duties  and 
responsibilities  and  the  tools  available  to  perform  their 
job  Once  the  SSM  position  is  described,  the  education  and 
training  requirements  necessary  to  perform  tho  job 
effectively  are  presented.  Next,  a  list  of  currently 
available  training  courses  that  meet  the  training 
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requlramants  Is  presented.  Finally,  a  3>year  SSM  training 
program,  encompassing  both  hardware  and  software.  Is 
recommended . 


fittEMnt  agM  TraAnAnq 

This  analysis  does  not  attempt  to  determine  individual 
SSM  training  needs,  but  instead  focuses  on  the  training 
needs  of  the  entire  ASC  SSM  corps.  Table  9  summarizes  the 
current  qualification  and  training  status  of  SSMs  at  ASC. 


Table  9.  ASC  SSM  Qualification  and  Training  Status  (30) 


SSMs 

Military 

Civilian 

Fu] 

Quail 

Lly 

Lfied 

Meed 

Training 

HW 

HW  & 
SW 

HW 

SW 

Number 

31 

22 

9 

12 

■■ 

19 

30 

Percent 

100.0 

71.0 

29.0 

38.7 

3.2 

61.3 

96.8 

HV  ->  Hardwtr*  SW  ->  Software 


The  current  ASC  SSM  qualification  process  is  an  ad  hoc 
mixture  of  formal  training  courses  and  on-the-job  training. 
Most  of  the  training  focuses  on  the  hardware  aspects  of 
system  safety  and  therefore  does  not  produce  a  fully 
qualified  SSM  with  software  safety  skills.  An  SSM 'a  level 
of  qualification  is  currently  determined  by  subjective 
assessment  and  has  not,  historically,  taken  software  safety 
into  consideration. 
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Tha  need  for  an  SSM  training  program  is  evident 
because,  as  Table  9  shows,  over  61  percent  of  ASC 
SSMs  are  not  fully  qualified  in  hardware  oriented  system 
safety,  and  over  96  percent  are  not  fully  qualified  in 
software  system  safety.  Although  the  hardware  oriented 
training  need  is  large,  the  need  for  software  safety 
training  is  overwhelming.  Interviews  with  SSMs  indicate  the 
training  shortfall  is  almost  entirely  in  the  areas  of  the 
software  development  process  and  software  safety  assurance 
methods.  This  shortfall  represents  a  significant  near  term 
training  burden. 

Current  ASC  estimates  indicate  it  takes  approximately 
3  to  5  years  to  fully  qualify  a  new  SSM  assuming  no  prior 
system  safety  experience  (30) .  Table  9  shows  that  military 
SSMs  make  up  71  percent  of  the  SSM  force  at  ASC  while 
civilian  SSMs  comprise  only  29  percent  of  ASC's  SSM  corps. 
Civilian  SSMs,  however,  remain  in  their  jobs  an  average  of 
three  to  four  times  longer  than  their  military  counterparts 
and  as  a  result,  represent  a  much  smaller  training  burden 
(30) .  Military  SSMs,  on  the  other  hand,  are  assigned  for 
only  3  to  5  years  and  will  most  likely  not  work  as  SSMs  in 
their  subsequent  assignments.  The  turnover  rate  for 
military  SSMs  at  ASC  la  approximately  five  par  year  (30) ; 
these  turnovers  represent  the  bulk  of  the  long  term  training 
load  and  support  the  need  for  an  on-going,  consolidated 
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training  program  as  the  primary  means  of  maintaining  fully 
qualified  SSMs. 


The  SSM«s  Job 

Before  a  training  and  education  recommendation  can  be 
made,  the  requirements  of  the  SSM's  job  need  to  be 
understood.  The  following  SSN  job  description  is  based  on 
bur  evaluation  of  ASC  local  training  materials,  policy  and 
guidance  documents,  and  personal  interviews  with  ssMs. 

% 

According  to  ASC  System  Safety  training  materials, 

system  safety  management  is  defined  as  the  process  that 

. . .  provides  the  optimum  degree  of  safety  within  the 
constraints  of  operational  effectiveness,  schedule,  and 
cost  through  timely  application  of  system  safety 
management  and  engineering  principles  whereby  hazards 
are  identified  and  corrective  actions  applied  to 
minimize  [safety]  risk  ....  (29) 

The  Program  Manager  Vs  System  Safety  Guide  indicates  that  the 

role  of  the  SSM  is  to  provide  a  competent  and  responsible 

"safety  overview  of  the  technical  and  management  aspects  of 

the  entire  program"  (10:33).  Therefore,  the  SSM  is  the 

primary  point  of  contact  for  all  system  safety  matters, 

management  and  technical,  within  a  system  program  office. 

The  SSM'S  day-to-day  job  consists  of  many  specific 

duties  which  are  listed  in  references  (10:7),  (11:25-27), 

and  (29) .  The  duties  involve  both  management  and  teohnical 

tasks  and  require  the  SSM  to  maintain  a  oontinuous  awareness 

of  system  safety  program  requirements,  design  and 

development  issues,  and  contractual  isnues  affecting  the 
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safety  of  the  syetein.  Management  duties  Include 

•  Remaining  abreant  of  system  safety  policies  and 
regulatory  requirements. 

•  Reviewing  program  management  documents  for 
Inclusion  of  relevant  system  safety  requirements. 

•  Ensuring  appropriate  system  safety  requirements 
and  tasks  are  included  In  procurement  documents. 

•  Determining  what  safety  critical  Issues  will  be 
addressed  at  program  reviews  and  what  contractual 
changes  may  be  required  to  address  emerging 
safety-critical  Items. 

Technical  duties  Include 

•  Reviewing  and  evaluating  the  contractor's  System 
Safety  Management  Plan  for  adequacy  of  system 
safety  analysis,  design,  and  test  procedures. 

•  Evaluating  and  monitoring  contractor  performance 
In  system  safety  analysis,  design,  and  test. 

•  Initiating  system  safety  programs  to  ensure  safety 
hazards  are  eliminated  or  sufficiently  controlled. 

While  It  Is  the  system  program  manager's  ultimate 
responsibility  to  ensure  that  the  safest  possible  system  Is 
developed,  the  program  manager  provides  the  SSM  with  the 
necessary  authority  to  perform  the  system  safety  management 
functions  (10:33).  The  SSM,  In  turn,  performs  the 
day-to-day  system  safety  management  functions  and  keeps  the 
program  manager  Informed  of  relevant  system  safety  Issues. 


Tha  SSM'B  TPOlE 

There  are  many  tools  available  to  the  SSM  to  assist  In 
performing  system  safety  related  responsibilities.  These 
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tools  include 

•  The  SSM's  training  and  education. 

•  System  safety  guidance  and  policy  documents. 

•  System  safety  analysis  and  risk  assessment 
reports. 

•  Program  reviews  and  system  safety  working  groups. 

The  most  Important  tool  that  an  SSM  possesses  Is  a  good 

education  and  training  l^aokground.  The  SSM  works  with  many 
people  having  both  technical  and  management  Interests  and 
must  project  a  self-confident  and  credible  Image.  While 

many  personal  qualities  can  contribute  to  credibility,  SSMs 

,  1 

must  possess  a  thorough  knowledge  of  their  job  that  comes 
only  from  having  the  proper  education  and  training. 

System  safety  guidance  and  policy  documents  tell  the 
SSM  what  must  be  done  to  Insure  development  of  a  safe  weapon 
system.  These  documents  form  the  basis  for  system  safety 
requirements  which  must  be  levied  on  the  developing 
contractor.  Contractor  compliance  with  safety  requirements 
and  the  elimination  or  control  of  safety  hazards  can  be 
verified  by  the  SSM  through  evaluation  of  system  safety 
analysis  and  risk  assessment  reports,  system  safety 
managers  coordinate  with  system  and  specialty  engineers 
within  the  program  office  to  verify  the  technical  accuracy 
of  the  analyses  and  reports.  Program  reviews  and  working 
groups  provide  the  SSM  with  feedback  to  gauge  the  status  of 
system  safety  efforts  as  well  as  offering  a  forum  for 
face-to-face  problem  Identification  and  resolution. 
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iaflL..sgii.ii.  Trttlning...flnd,..EdusfltifiD 


The  training  and  education  requirements  for  SSMs  were 
derived  by  reviewing  safety  hazard  analysis  and  risk 
assessment  reports ,  interviewing  SSMs,  and  considering  the 
86M  job  description  from  the  previous  section.  The  training 
requirements  for  an  SSM  fall  into  two  categories.  First, 
there  are  requirements  that  Involve  general  system  and 
safety  related  knowledge.  They  represent  skills  that  apply 
to  the  system  safety  management  of  any  system  regardless  of 
its  mix  of  hardware  and  software.  Second,  there  are 
training  requirements  directly  related  to  software 
development  and  its  impact  on  system  safety.  SSM  training 
requirements  are  shown  in  Table  10  as  an  enumerated  list  to 
facilitate  cross-referencing  to  detailed  requirements 
discussion  and  available  training. 

Review  of  hazard  doouments  revealed  technical  skills 
that  an  SSM  should  have  knowledge  of  in  order  to  understand 
and  evaluate  hazard  analyses  and  risk  assessments.  As  a 
whole,  the  document  review  was  extremely  helpful  in 
identifying  methodologies,  skills  and  analysis  techniques 
relating  to  system  safety.  The  treatment  of  software 
safety,  however,  in  the  doouments  reviewed  was 
disappointing.  None  of  the  doouments  included  a  detailed 
software  safety  analysis,  one  analysis  virtually  dismissed 
software  as  a  system  safety  issue  while  another  merely 
included  a  few  generic  software  design  issues  taken  from  a 
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Tabl*  10.  S8M  Training  Raqulramanta 


Training  Raqulremants 

1.  System  Acquluttlon  Procass _ 

2.  Sytyani  Davalopaant  Proq'»a« 

3.  Syatan  Safaty _ 

a)  Quldanea  and  Policy _ 

b)  Managamant _ _ 

c)  Analyala _ 

d)  Daalgn 

a)  Taatlng _ 

4.  Rlik  AiBaaimant 

- . . .  I  '  . - . . 

5.  Syatam  Utidaratandlng _ 

6.  Softvara  AcquUttlon  Llfa-Cycla 

7.  Softvata  Davalopmant  Procaaa _ 

8.  Soffttwara  Taat  and  Evaluation 

9.  Softwara  Safaty  Analyala  Taohnlquea 


Boftwara  davalopmant  handbook.  Howavari  tha  Softwara 
Ragulramanta  Hazard  Analyala  obtalnad  from  tha  Elaotronlo 
Combat  and  RaoonnalBsanaa  Syetam  Program  Office  employed  the 
FKA  methodology  to  provide  an  Initial  look  at  tha  safety 
hazards  assoolatad  with  softwara  raqulraments. 

intarviawB  with  SSMs  oorroboratad  the  teohnloal 
training  raqulraments  and  provided  tha  primary  means  for 
determining  management  skills  an  SSN  should  have  In  order  to 
effectively  manage  a  system  safaty  program.  Tha  degree  of 
understanding  required  for  any  specific  training  area  was 
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detexrmlnttd  by  comparing  spaolflc  skills  and  prlnclplss  to 
ths  SSM  job  dssoriptlon.  Finally,  the  educational 
background  of  an  SSM  was  determined  by  assessing  the 
specific  skills  and  principles  encountered  In  the  training 
requirements. 

trainlna  Reemlrements .  An  SSM  should  have  training  in 
each  0^  the  following  general  areas: 

1.  System  aocmlsltion  process.  An  SSM  should 
understand  the  basic  mechanics  of  system  acquisition  at  both 
the  DOD  and  program  office  levels.  At  the  dod  level,  an  SSM 
should  understand  the  major  milestones  In  the  acquisition 
process  and  know  how  a  program  evolves  from  one  milestone  to 
the  next.  At  the  program  office  level,  an  SSM  must 
understand  day-to-day  acquisition  functions  such  as 
development  of  requirements  and  specifications;  evaluation 
of  contractor  proposals  and  plans;  and  reviewing  and 
approving  contractor  submitted  data.  Interviews  with  SSMs 
Indicate  that  knowing  where  a  system  is  In  its  particular 
life-cycle  and  knowing  what  details  that  stage  of  the 
life-cycle  entails  allow  the  SSM  to  address  system  safety  at 
an  appropriate  level.  Additionally,  the  SSM  who  understands 
program  office  acquisition  functions  can  become  a  competent 
and  trusted  member  of  the  program  office  team  and  Impact 
system  development  with  appropriate  system  safety 
requirements. 


eo 


2.  SvBtam  dttvlQDwnt  arQQ»as.  An  SSM  should 


know  how  a  syatsin  is  dsvslopsd  and  understand  how  a  system 
developer  goes  through  the  process  of  requirements  analysis, 
making  engineering  trade-offs,  designing,  building,  and 
testing.  Additionally,  the  SSM  should  understand  how 
hardware  and  software  development  efforts  differ  and  how 
their  eventual  integration  into  a  system  affects  the  safety 
of  the  overall  system.  SSM  interviews  indicated  that 
understanding  the  development  process  is  vital  to  an  SSM's 
ability  to  evaluate  and  direct  the  contractor. 

3.  System  safety.  The  SSM  should  thoroughly 
understand  how  to  Implement  and  manage  a  system  safety 
program.  An  SSM  needs  training  in  the  areas  of  safety 
guidance  and  policy,  management,  analysis,  design,  and 
testing. 

a.  guidance  and  Policy.  The  SSM  must  know 
which  directives,  policies,  regulations,  and  handbooks 
provide  guidance  and  requirements  on  system  safety  issues. 
An  SSM  must  be  able  to  evaluate  the  precedence, 
applicability,  and  system  impacts  of  the  many  system  safety 
guidance  and  policy  sources.  Interviews  with  SSMs  indicate 
that  knowing  system  safety  guidance  and  policy  allows  the 
SSM  to  place  the  proper  requirements  on  contract. 

b.  Management.  Thu  SSM  must  be  able  to 
manage  Informally  as  well  as  apply  formal  system  safety 
management  tools.  The  SSM  needs  to  understand  MIL-STD-8S2B 
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fully  and  ba  able  to  aelaot  appropriate  taaks  to  plan, 
conduct,  and  evaluate  a  system  safety  program.  Interviews 
with  SSMs  Indicate  that  an  SSM  must  be  able  to  review  and 
critique  the  System  Safety  Program  Plan  (SSPP)  as  well  as 
any  of  the  hazard  analyses  and  assessments  employed  on  a 
particular  development  effort. 

e.  Analysis.  An  SSM  should  pe  familiar  with 
system  safety  analysis  techniques  used  by  Industry.  A 
rudimentary  understanding  6f  safety  analysis  techniques  Is 
required  to  evaluate  and  critique  contractors'  safety 
analyses.  This  requirement  Is  supported  by  the  fact  that 
all  of  the  hazard  analysis  reports  reviewed  employed  some 
form  of  Fault  Hazard  Analysis  (FHA)  to  address  software 
safety.  In  addition,  the  F<-16  System  Safety  Hazard  Analysis 
employed  a  detailed  hardware  based  Fault  Tree  Analysis 
(FTA).  The  F-22  analyses  selectively  employed  FTA  in 
determining  event  probabilities.  Finally,  the  C-17  analyses 
employed  FTA  and  Sneak  Circuit  Analysis  on  selected  safety 
critical  subsystems. 

_ Design.  The  SSM  should  be  familiar  with 

safety  design  techniques  and  methodologies.  SSMs  should  be 
aware  of  how  safety  risks  posed  by  a  particular  weapon 
system  can  be  avoided  or  controlled  to  an  acceptable  level. 

e.  Testing.  The  SSM  should  understand  both 
the  development  and  operational  test  and  evaluation 
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prooegBss  for  wtapon  BygtainB.  An  SSM  nesds  to  understand 

•  the  llnltB  of  testing r 

•  formulation  of  testing  requirements; 

•  and  principles  of  validation  and  verification. 
Syatem  safety  guidance  documents  indloate  that  "...  safety 
requirements  need  to  be  verified  by  analysis,  inspection, 
demonstration,  or  test"  (8tA-l5). 

4.  Risk  assessment.  The  SSM  should  understand 
how  system  safety  risk  assessments  are  performed  as  well  as 
have  a  basic  understanding  of  the  quantitative  analysis  of 
event  probabilities.  In  addition,  the  SSM  needs  to 
understand  how  system  reliability  is  determined  and  how 
reliability  information  Is  Integrated  Into  the  system  safety 
risk  assessment., 

s.  System  understanding.  An  SSM  must  be  familiar 
with  the  system  under  development.  Every  system  safety 
analysis  report  reviewed  starts  with  a  system  description 
which  serves  as  the  basis  for  the  analysis.  Interviews  with 
SSMs  Indloate  a  working  knowledge  of  the  Individual 
subsystems  that  comprise  a  weapon  system  as  well  as  the 
Integrated  whole  weapon  system  Is  needed. 

&j _ lbi-iogAMagd..flBguUitifln..3,igE-gyBli.  The  ssm 

should  understand  the  oharaoterlstlos  of  the  software 
llfe-oyole  In  terms  of  software  development  and  maintenance. 
Understanding  that  software  does  not  break  or  fall  In  the 
traditional  hardware  sense  Is  Important  in  determining  the 
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reliability  and  performing  failure  mode  analyeie  of  a 
software  intensive  system* 

It _ The  software  development  process.  An  SSM  must 

understand  how  software  is  designed,  built,  and  tested. 
Speolflo  knowledge  of  the  design,  code,  testing,  dnd 
redesign  cycle  for  a  computer  Software  unit,  Computer 
Software  Component,  an^  Computer,  Software  Configuration  Item 
development  will  aid  the  SSM  in  determining  how  safety 
impacts  the  software  development  process. 

fij _ flggtwttca.  ■t«ttL,.ftnii..gval,uaAioji  >  The  ssm  must  be 

familiar  with  ba^eic  approaches  to  software  testing  and 
evaluation.  The  sqm  must  understand  that  exhaustive  testing 
of  software  is  virtually  impossible,  and  therefore,  the  need 
for  careful  design  and  analysis  of  software  testing  is 
critical  to  ensuring  software  system  safety. 

SL _ Software  safety  analysis  teohnicfues.  The  SSM 

must  stay  up  to  date  on  current  software  safety  analysis 
techniques  as  well  as  techniques  that  are  under  development. 
The  SSM  must  be  in  a  position  to  lead  the  contractor  and 
suggest  possible  software  safety  analysis  techniques  as 
there  is  currently  no  standard  method  to  perform  software 
system  safety  analysis  and  assessment  within  the  DOD  and 
industry. 

Education  Requirements.  Based  on  the  levels  and  types 
of  training  identified  above,  the  minimum  education 
requirement  for  an  SSM  should  be  a  bachelor  of  science 
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degree  in  engineering  or  some  related  applied  eolenoe.  To 
evaluate  and  direct  the  work  of  both  system  and  safety 
engineers,  the  SSN  should  be  conversant  In  both  the  language 
and  techniques  of  the  scientific  community. 

Based  on  the  above  training  and  education  requirements 
we  believe  the  Ideal  SSM  should  be  a  technically  oriented 
person  with  at  least  a  basic  understanding  of  the  DOD 
acquisition  development  process  (hardware  and  software) ; 
system  and  software  safety  guidance  and  policy  documents; 
and,  as  governed  by  their  program,  the  applicable  hardware 
and  software  safety  analysis  techniques  In  use  by  their 
contractor. 

AY.ailBblt  Training 

The  best  way  for  an  SSM  to  become  fully  qualified  Is 
through  the  proper  combination  of  formal  training  ooursee 
and  on-the-job  training.  The  following  list  presents  the 
available  training  courses  that  can  help  meet  the  formal 
training  needs  of  an  SSN. 

ASC  Training  Course: 

1.  ASC  system  Safety  Training  course  -  conducted 
locally  by  the  ASC  system  Safety  Office 

Air  Force  Sponsored  Courses: 

2.  WSYS  100  -  Introduction  to  Acquisition 
Management  (or  Its  equivalent  SAS  001) 

3.  WSYS  200  -  Acquisition  Planning  and  Analysis 
(or  its  equivalent  SAS  006) 

4.  SAS  002  -  Computer  Hesources  Acquisition 
Course 
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5. 


WSYS  212  -  Mission  Critical  Computer  Software 
Project  Management 

6.  wsys  229  -  Test  and  Evaluation  Management 

7.  WCIP  057  -  Systems  Safety  Management  Course 

8.  WCIP  060  -  System  Safety  Analysis  Course 

9.  WCSE  471  -  Software  Engineering  concepts 

10.  WCSE  472  -  specification  of  Software  Systems 

11.  WCSE  473  -  Principles  and  Applications  of 
Software  Design 

12.  WCSE  474  -  Software  Generation  and 
Maintenance 

13.  WCSE  475  -  Software  Verification  and 
Validation 

14.  WQMT  020  -  Reliability  and  Maintainability 
Overview 

15.  WQMT  335  -  Reliability  and  Maintainability 
Design  in  Systems  Acquisition 

Navy  Sponsored  Courses: 

16.  SS-320  -  Systems  Safety  Concepts 

17.  SS-510  -  Systems  Safety  and  Human  Factors 

18.  88-520  -  Advanced  8ystems  safety  Techniques 

Commercially  Available  Courses: 

19.  System  Safety  Course  -  available  through  the 
University  of  Southern  California 

20.  Software  Safety  Course  -  available  through 
the  University  of  Southern  California 

21.  Software  Safety  Workshop  -  available  through 
the  University  of  Californla-Los  Angeles 

Evaluating  the  above  courses  helped  to  determine  which 
training  requirements  are  met  by  each  course.  The 
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evaluations  were  based  on  course  descriptions  contained  In 
the  1  September  1991  version  of  APR  50-5,  Training;  USAF 
Formal  Schools,  current  course  syllabi,  and  interviews  with 
SSMs.  For  more  information  on  the  courses  listed  above 
refer  to  Appendix  H. 

The  results  of  the  course  to  requirements  evaluations 
are  compiled  in  Table  11  and  Table  12.  An  'X'  in  the  table 
indicates  that  the  minimum  training  requirement  is  met. 
Although  several  of  the  requirements  are  met  by  numerous 
courses,  it  is  the  opinion  of  the  researchers  that  each 
subsequent  course  (within  a  sequence)  builds  on  concepts 
learned  in  the  previous  course  and  further  enhances  subject 
understanding. 

All  of  the  training  requirements  listed  in  Table  ll  and 
Table  12,  except  system  understanding,  can  be  met  by  several 
course  combinations.  System  understanding,  however,  is  a 
requirement  that  is  conceptually  different  for  each  SSM.  it 
relates  to  the  particular  weapon  system  or  subsystem  under 
development  and  is  one  of  the  most  important  aspects  of  an 
SSM's  job.  SSMs  must  assume  the  responsibility  to  become 
familiar  with  their  systems.  Formal  training  cannot 
compensate  for  a  lack  of  system  understanding. 
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Table  11-  Training  Reguirement:s  Matrix 
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9.  SSAT< 


SSMb  are  responsible  for  making  safety-crltlcal 
decisions  relating  to  system  and  software  safety  that  may 
prevent  loss  of  life  or  damage  to  equipment.  Many  of  these 
decisions  are  based  on  the  review  of  hazard  analyses  and 
risk  assessments  submitted  by  the  developing  contractor.  To 
effectively  evaluate  the  analysis  reports  an  must  be 
adequately  trained. 

As  discussed  in  the  previous  section,  both  short  and 
long  term  SSM  training  needs  must  be  met.  Many  of  the  SSMs 
interviewed  think  they  would  benefit  from  additional 
training,  speoifioally  in  the  area  of  software  development 
and  safety  analysis.  In  addition,  there  is  a  consensus 
among  SSMs  on  the  need  for  an  introductory  level 
consolidated  training  program  for  newly  assigned  SSMs. 

To  address  the  need  for  a  consolidated  training  program 
that  provides  an  SSM  with  a  complete  system  (including  both 
hardware  and  software)  safety  management  capability,  a 
training  program  Including  the  curriculum  listed  in  Table  13 
is  recommended.  The  program  outlined  represents  a  minimum 
set  of  existing  courses  and  involves  a  number  of  assumptions 
concerning  the  trainee,  course  content,  and  the  training 
environment. 

The  best  way  to  outline  a  comprehensive  training 
program  is  to  assume  the  trainee  is  a  newly  hired  individual 
with  only  an  undergraduate  degree  in  engineering  or  applied 
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Table  13.  Recommended  Training  Curriculum 


Course  Number 

Title 

ASC  Local 
Course 

Introduction  to  System  Safety  and 
Acquisition  Management 

WSYS  100 

Introduction  to  Acquisition  Management 

WSYS  200 

Acquisition  Planning  and  Analysis 

SAS  002 

Computer  Resources  Acquisition  Course 

WCIP  057 

Systems  Safety  Management  Course 

WCIP  060 

System  Safety  Analysis  Course 

WCSE  471 

Software  Engineering  Concepts 

WQMT  020 

Reliability  and  Maintainability  Overview 

SS  510 

Systems  Safety  and  Human  Factors 

SS  520 

Advanced  Systems  Safety  Techniques 

SPT  YY-# 

Software  Safety  Course  @  USC 

ENG  619,230 

Software  Safety  Workshop  @  UCLA 

uolenoe.  since  not  all  SSMe  come  to  the  job  void  of  safety 
related  experience,  the  recommended  program  can  be  easily 
modified  to  accommodate  individual  training  needs. 

Based  on  the  course  to  requirements  evaluations 
conducted  previously,  the  training  courses  included  in  this 
program  meet  or  exceed  the  minimum  training  requirements. 
Other  courses  listed  in  Table  11  and  Table  12  may  be  added 
to  meet  a  specific  training  need  or  to  enhance  an  SSM's 
general  knowledge. 

The  training  environment  is  assumed  to  be  ideal  in  that 
required  courses,  training  time,  and  training  funds  are 
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available  when  needed.  These  basic  assumptions  allow  us  to 
present  the  following  training  program. 

Figure  9  shows  a  3-year  training  program  whicl>  j/ocluces 
all  courses  listed  in  Table  13.  The  courses  are  o:::hedulHd 
to  insure  that  prereq[ui8ite8  will  be  met.  The  schedule  also 
shows  liberal  amounts  of  slack  time  between  courses  to 


Figure  9.  Recommended  SSM  Training  Program 


conduct  on-the-job  training  or  assimilate  course  material. 
The  allocation  of  slack  time  in  Figure  9  is  purely 
discretionary.  The  3-year  schedule  allows  for  a  mix  of 
formal  and  on-the-job  training.  As  the  SSM  gains 
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operational  experience,  advanced  software  safety  management 
skills  are  presented  through  formal  training  courses. 

The  first  year  of  training  is  designed  to  produce  a 
functionally  qualified  SSM  who  oan  perform  with  minimum 
supervision.  The  ASC  local  system  safety  course  provides  an 
Introduction  to  system  acquisition,  system  development,  and 
system  safety,  WSfS  100  provides  a  more  detailed  look  at 
system  acquisition  and  development,  WQMT  020  provides  an 
overview  of  system  reliability  and  maintenance.  WCIP  057 
addresses  system  engineering  and  management  and  provides  a 
more  detailed  look  at  system  safety  management  methods. 

SAS  002  addresses  software  acquisition,  the  software 
development  process,  and  an  Introduction  to  the  Ada 
programming  language.  Finally,  WCSE  471  provides  a  look  at 
software  engineering  and  development  from  requirements 
analysis  to  support  and  maintenance. 

The  second  year  emphasizes  system  safety  analysis 
methods.  WCIP  060  provides  an  Introduction  to  software  and 
hardware  safety  analysis  techniques,  probability  theory,  and 
risk  assessment.  SS-510  introduces  system  safety  and  human 
factors  engineering  techniques.  SS~520  provides  a  detailed 
look  at  system  safety  analytical  techniques  Including 
software  safety  analysis  techniques. 

Finally,  the  third  year  of  training  completes  the 
course  work  and  on-the-job  training  necessary  to  produce  a 
fully  qualified  SSM.  WSYS  200  provides  detailed  training  on 
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program  planning,  executing,  and  controlling.  As  a  final 
course,  either  the  USC  course  or  the  UCLA  workshop  should  be 
taken  to  provide  an  understanding  of  current  software  safety 
analysis  trends  In  Industry  and  academia. 

We  believe  the  above  training  program  Is  realistic  and 
achievable  because  It  Is  based  on  existing  and  available 
training  courses.  The  training  program  Is  designed  to 
produce  a  functionally  qualified  SSM  In  one  year  and  a  fully 
qualified  SSM  by  the  end  of  the  third  year.  Training  time 
and  qualification  ratings  are  based  on  the  assumption  that 
the  SSM  has  no  prior  acquisition  or  safety  experience. 

Conclusion 

This  analysis  examined  the  education  and  training 
requirements  necessary  for  SSNs  to  perform  their  job 
effectively.  By  examining  the  functions  of  the  SSM  position 
It  Is  evident  that  the  SSM  is  an  Integral  part  of  the  system 
development  team.  However,  the  education  and  training 
required  to  qualify  an  SSM,  to  date,  have  been  predominately 
hardware  oriented.  SSNs  must  have  system  and  safety 
knowledge  that  Includes  both  hardware  and  software  If  they 
are  going  to  accurately  and  effectively  manage  a  system 
safety  program.  As  software  comes  to  dominate  the 
acquisition  of  high  technology  weapon  systems,  training  that 
will  allow  SSMs  to  effectively  manage  hardware  and  software 
safety  programs  must  be  provided. 
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YL _ Conoluslons  and _RecQinmendat Ions 


The  preceding  examination  of  software  safety  In  DOD 
weapon  system  development  established  the  relationships 
between  the  DOD  software  development  process,  DOD  system 
safety  program  requirements  and  software  safety  analysis 
techniques.  The  combined  results  from  the  literature 
review,  telephone  survey,  and  personal  interviews  then 
pointed  out  how  these  three  components  combine  to  provide 
some  level  of  assurance  that  the  system  under  development 
will  operate  at  an  acceptable  level  of  risk. 

Given  the  complexities  of  the  interrelationships  among 
the  components  of  a  software  safety  program,  we  analyzed  the 
training  and  education  requirements  necessary  to  prepare  DOD 
System  Safety  Managers  (SSM)  to  implement  and  manage  a 
system  safety  program  effectively.  A  consolidated  training 
program  was  developed  and  presented  in  Chapter  V. 

The  next  section  of  this  chapter  discusses  conclusions 
reached  following  the  accomplishment  of  the  stated  research 
objectives.  Recommendations  that  should  be  Implemented  to 
enhance  the  DOD 'a  position  in  the  field  of  system  and 
software  safety  follow.  Finally,  recommendations  for 
further  study  are  presented. 
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The  following  conclusions  were  reached  as  a  result  of 
accomplishing  the  research  objectives  for  this  study. 

1.  The  framework  for  implementing  and  managing  an 
effective  POD  weapon  system  software  safety  program  exists. 
The  software  development  process  defined  in  DOD-STD-2167A  is 
detailed  and  highly  structured.  MIL-STD-1251B  defines 
formal  reviews  and  audits  that  provide  ample  opportunity  to 
discuss  specific  system  and  software  safety  issues.  In 
addition,  MIL-STD-882B  defines  system  and  software  safety 
analysis  reqtuirements .  Finally,  structured  analysis 
techniques  have  been  developed  or  adapted  specifically  for 
software  safety  analysis  and  can  provide  important  software 
hazard  Information  when  applied  throughout  the  software 
development  process. 

2.  in  practice,  software  safety  programs  are  Jiot 
implemented  and  managed  effectively.  The  principal 
limitation  of  the  software  development  process,  from  a 
safety  standpoint,  is  that  potentially  all  of  the  safety 
review  can  be  tailored  out  of  the  process.  System  safety 
program  requirements  are  also  allowed  to  be  tailored  by  DOD 
program  managers.  Often,  the  system-level  safety  analysis 
(200  Series)  tasks  of  MIL-STD-8B2B  are  chosen  over  the  more 
specific  software  safety  analysis  (300  Series)  tasks.  In 
response,  developers  use  hardware  oriented  safety  analyses 
that  inadequately  address  software  safety.  In  addition. 
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many  development  organizations  have  not  integrated  software 
safety  into  their  overall  software  development  process. 
Finally,  there  is  no  known  standard  management  methodology 
for  software  safety  in  industry  or  the  DOD. 

3.  gtrugturftd,  BQfWflrg  gaiety  anail.YJBi,B.-tgghnimi9g-.ttrg, 

not  widely  used  throughout  the  aerospace  industry  and  DOD. 


The  most  common  analysis  approach  is  the  Fault  Hazard 
Analysis  methodology.  This  is  an  inductive  analysis  method 
that  results  in  a  qualitative  safety  analysis.  Most  other 
techniques  are  deductive  in  nature  and  share  the  common 
characteristics  of  being  difficult  and  expensive  to  apply. 

In  addition,  the  application  of  these  techniques  has  been 
limited  to  small  software  systems.  Finally,  DOD  has  not 
placed  a  demand  on  industry  to  apply  any  of  these  techniques 
and  the  general  consensus  among  developers  was  why  spend 
time  and  money  on  obtaining  structured  software  safety 
analysis  capability  that  the  government  has  not  asked  for! 

4 .  The  Air  Force  needs  a  consolidated  training  proari 
for  System  Safety  Managers  that  covers  both  hardware  and 
software  system  safety.  Over  96  percent  of  Aeronautical 
System  Center's  System  Safety  Managers  are  not  fully 
qualified  in  software  system  safety.  SSMs  are  responsible 
for  implementing  and  managing  system  safety  programs  that 
address  both  hardware  and  software  safety,  currently,  few 
SSMs  fully  understand  software  development  and  software 
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safety  analysis  and  their  training  has  been  both  ad  hoc  and 
hardware  oriented. 


Recommendations  For  Implementation 

Based  on  the  conclusions  of  this  study,  the  foil  Lng 
recommendations  will  increase  the  DOD's  effectiveness  in 
implementing  and  managing  system  and  software  safety 
programs . 

1.  To  provide  Air  Force  SSMs  with  the  knowledge 
necessary  to  effectively  implement  and  manage  a 
Gomprehenslve  system  safety  program,  the  training  program 
presented  in  Chapter  V  should  be  Implemented. 

2.  Many  aerospace  contractors  conduct  in-house  system 
and  software  safety  training  programs  for  their  engineers 
and  managers.  DOD  system  safety  engineers  and  managers 
should  attend  these  training  programs  to  increase  their 
familiarity  with  the  safety  techniques  and  methodologies 
being  used  to  develop  their  systems. 

3.  To  provide  Air  Force  engineers  and  managers  with 
system  and  software  safety  familiarity,  the  Air  Force 
Institute  of  Technology  6hou''d  Include  in  its  advanced 
degree  and  professional  continuing  education  currioulums  a 
block  of  instruction  covering  system  and  software  safety. 
Target  currioulums  should  include  computer  engineering, 
software  engineering,  systems  engineering,  systems 
management,  software  systems  management,  and  information 
resource  management.  System  and  software  safety  instruction 
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should  provide  familiarity  with  current  DOD  system  safety 
policy  and  guidance  and  the  relationships  between  the 
development  process,  system  safety  program  requirements,  and 
safety  analysis  techniques. 

4.  To  help  establish  a  standard  management  methodology 
for  software  safety  programs,  the  Air  Force  should  implement 
a  command-level  System  and  Software  Safety  Working  Group. 
This  group  should  meet  blannually  to  dlsouBs  system  and 
software  safety  implementation  and  management  issues  in 
regards  to  weapon  system  development.  This  group  can  also 
provide  a  forum  for  MIL-STD-882  evaluation  and  establish  a 
system  and  software  safety  lessons  learned  database. 

5.  DOD  system  safety  engineers  and  managers  should 
actively  participate  in  Industry  forums  for  system  and 
software  safety.  The  institute  of  Electrical  and 
Electronics  Engineers  (IEEE)  computer  Assurance  (COMPASS) 
Conference  and  the  Electronics  Industry  Association  (EIA) 
are  two  prominent  forums  that  focus  on  software  safety. 
Presenting  papers  and  participation  in  panel  discussions  at 
these  forums  provides  an  excellent  opportunity  for  DOD 
representatives  to  influence  Industry's  refinement  of  old, 
or  development  of  new,  software  safety  analysis  techniques 
and  methodologies. 

EagtfminflndatignB  Jlflic  rurthar  study 

Based  on  the  conclusions  reached  in  this  study,  the 
following  recommendations  for  further  study  will  Increase 
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the  DOD's  knowledge  in  Implementing  and  managing  system  and 
software  safety  programs. 

1.  study  System  Safety  Manager  jobs  In  the  Army,  Navy, 
and  other  government  agencies.  The  objective  would  be  to 
learn  what  qualification  and  training  requirements,  training 
programs,  and  training  courses  are  used  by  these  agencies 
and  how  the  Air  Force  and  DOD  could  benefit  from  them. 

2.  Develop  a  prototype  database  of  software  system 
safety  hazards  and  risks  that  can  be  used  by  system  and 
safety  engineers  to  start  a  software  safety  analysis  effort. 
There  are  numerous  lessons  learned  databases  and  design 
handbooks  throughout  the  DOD  that  address  safety  items. 

They  are,  however,  not  integrated.  This  task  would  glean 
software  safety  hazards,  risks,  and  guidelines  from  various 
sources  to  be  cataloged  in  a  database. 

3.  Select  a  structured  software  safety  analysis 
technique  and  apply  it  to  a  software  module  of  reasonable 
size.  Items  to  be  studied  might  include  what  safety 
analysis  data  must  be  known  to  start  the  structured 
analysis,  how  hard  or  easy  is  the  technique  to  learn  and 
apply,  and  how  large  can  the  software  module  be  before  the 
analysis  becomes  intractable. 

4.  Track  and  evaluate  the  hazard  analyses  and  risk 
assessments  resulting  from  the  300  series  tasks  of 
MIL’-STD-882B  currently  on  contract.  This  study  should  focus 
on  the  specific  techniques  and  methodologies  used  to  conduct 
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the  tasks  and  measure  the  effectiveness  of  the  analysis 
obtained. 


In  addition  to  the  above  recommendations  for  further 
study,  the  following  topics  for  study,  although  not  directly 
supported  by  this  research,  can  expand  the  DOD's  knowledge 
of  system  and  software  safety  management. 

5.  Track  the  application  and  resulting  software  safety 
analyses  of  MIL-ST0-882C.  MIL-STD>882C  is  due  for 
distribution  in  December  1992  and  will  combine  the  200  and 
300  series  tasks  of  MIL‘~STD-882B  into  a  single  set  of  system 
safety  tasks. 

6.  Broaden  the  focus  of  this  study  by  investigating 
more  diverse  applications  of  software  system  safety 
techniques  in  such  fields  as  medical  equipment  development, 
nuclear  weapons  development,  nuclear  power  systems,  mass 
transit  systems,  etc. 

7.  Expand  the  understanding  and  practice  of  software 
safety  analysis  in  the  United  States  by  studying  how  other 
countries  like  Great  Britain,  Australia,  and  Japan  ensure 
the  safety  of  their  software  systems. 
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AggQnymfl 

AFMC  -  Air  Force  Materiel  Command 

ASC  -  Aeronautical  Syeteme  Center 

CCB  -  Change  Control  Board 

CDR  -  Critical  Design  Review 

CDRL  -  Contractor  Data  Regulrementa  Liat 

CSC  -  Computer  Software  Component 

CSCI  -  Computer  Software  Configuration  Item 

CSU  -  Computer  Software  Unit 

DOD  -  Department  of  Defense 

FCA  -  Functional  Configuration  Audit 

FHA  -  Fault  Hazard  Analysis 

FMECA  -  Failure  Mode,  Effects,  and  Criticality  Analysis 

FQR  -  Formal  Quallfloatlon  Revitw 

FQT  -  Formal  Quallfloatlon  Testing 

FSED  -  Full  Scale  Engineering  Development 

GFE  -  Government  Furnished  Equipment 

GFP  -  Government  Furnished  Property 

LANTIRN  ~  Low  Altitude  Navigation  and  Targeting  Infrared  for 
Night 

NOA  -  Memorandum  of  Agreement 

NSCCA  -  Nuclear  Safety  Cross-Check  Analysis 

06SKA  -  Operating  &  Support  Hazard  Analysis 

PCA  -  Physical  Configuration  Audit 

PDR  -  Preliminary  Design  Review 
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PHA 

- 

Preliminary  Hazard  Analysis 

PHL 

- 

Preliminary  Hazard  List 

PMD 

- 

Program  Management  Directive 

PMP 

- 

Program  Management  Plan 

RTL 

- 

Real  Time  Loglo 

RW 

- 

Bleotronla  Combat  and  Reconnaissance 

SCHA 

- 

Software  Change  Hazard  Analysis 

SDF 

- 

Software  Development  Folder 

SDR 

- 

System  Design  Review 

SFHA 

- 

Software  Fault  Hazard  Analysis 

SFMEA 

- 

Software  Failure  Modes  and  Effects  Analysis 

SHA 

mm 

System  Hazard  Analysis 

SIP 

- 

System  Improvement  Program 

SOW 

- 

Statement  of  Work 

SPO 

- 

System  Program  Office 

SPS 

- 

Software  Product  Specification 

SRA 

- 

Software  Req[ulrements  Analysis 

SRHA 

- 

Software  Requirements  Hazard  Analysis 

SRR 

System  Req\.ilraments  Review 

SSCA 

- 

Software  Sneak  Circuit  Analysis 

SSG 

■r. 

System  Safety  Group 

SSHA 

Subsystem  Hazard  Analysis 

SSM 

- 

System  Safety  Manager 

SSPP 

- 

System  Safety  Program  Plan 

SSR 

- 

Software  Specification  Review 

STD 

Software  Test  Description 
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SWIM  -  System  Wide  Integrity  Management 

TRR  -  Test  Readiness  Review 

UCLA  -  University  of  California  -  Los  Angeles 
use  -  University  of  Southern  California 


Definitions 

Base  line.  A  configuration  Identification  document  or  a  set 
of  such  documents  formally  designated  and  fixed  at  a 
specific  time  during  a  Cl's  life  cycle.  Base  lines,  plus 
approved  changes  from  those  base  lines,  constitute  the 
current  configuration  identification.  For  configuration 
management  there  are  three  base  lines,  as  follows: 

b.  Allocated  base  line.  The  initial  approved 
allocated  configuration  identification. 

a.  Functional  base  line.  The  initial  approved 
functional  configuration  identification. 

0.  Product  base  line.  The  initial  approved  or 
conditionally  approved  product  configuration 
identification.  (6:61) 

Hazard.  A  set  of  conditions  (i.e.,  a  state)  that  can  lead 
to  an  accident  given  certain  environmental  [operating] 
conditions.  (25:223) 

Hazard  Probability.  The  aggregate  probability  of  occurrence 
of  the  individual  hazardous  events  that  create  a  specific 
hazard.  (8:2) 

Hazard  Severity.  An  assessment  of  the  worst  credible  mishap 
that  could  be  caused  by  a  specific  hazard.  (8:2) 

Mishap.  An  unplanned  event  or  series  of  events  that  leads 
to  an  unacceptable  loss  such  as  death,  injury,  Illness, 
damage  to  or  loss  of  equipment  or  property.  (8:2) 

Safety-Critical .  Those  software  operations  that,  if  not 
performed,  performed  out-of -sequence,  or  performed 
incorrectly  could  result  in  improper  control  functions  (or 
lack  of  control  functions  required  for  proper  system 
operation)  which  could  directly  or  indirectly  cause  or  allow 
a  hazardous  condition  to  exist.  (13:3) 
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Software  Reliability.  The  probability  that  the  software 
will  execute  for  a  particular  period  of  time  without  a 
failure (  weighted  by  the  cost  to  the  user  of  each  failure 
encountered.  (13:3) 

Software  Safety.  The  application  of  system  safety 
engineering  techniques  to  software  In  order  to  insure  and 
verify  that  the  software  design  takes  positive  measures  to 
enhance  system  safety  and  that  errors  which  could  reduce 
system  safety  have  been  eliminated  or  controlled  to  an 
acceptable  level  of  risk.  (13:3) 

software  Safety  Analysis.  A  hazard  evaluation  technique 
which  Identifies  software  commands  (with  and  without  errors) 
either  singularly  or  In  combination  with  hardware  responses 
that  could  result  In  safety  critical  hazards.  (13:3) 

software  Security.  The  degree  to  which  a  piece  of  software 
protects  itself y  or  is  protected,  from  unauthorized,  and 
sometimes  malicious,  actions.  (32:Appendix  1) 

System  Safety.  The  application  of  engineering  and 
management  principles,  criteria,  and  techniques  to  optimize 
safety  within  the  constraints  of  operational  effectiveness, 
time,  and  cost  throughout  all  phases  of  the  system  life 
cycle.  (8:3) 

Validation.  The  process  of  confirming  that  the  software 
(i.e,  documentation  and  computer  program)  satisfies  all  user 
requirements  when  operating  in  the  user  environment. 

(12SC-10) 

Verification.  The  process  of  confirming  that  the  products 
of  each  software  development  phase  (e.g.,  requirements 
definition,  design,  and  coding)  are  complete,  correct,  and 
consistent  with  reference  to  the  products  of  the  preceding 
phase.  (12:C-10) 
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Appendix  Bi  Guldanofl  and  Policy  Documents 

This  appendix  lists  guidance  and  policy  documents 
applicable  to  the  DOD  software  development  process  and 
system  safety  program.  This  list  Is  not  exhaustive  and 
represents  only  doouments  encountered  during  this  research. 

Software  Development  Prooeas 


DOD  DIR  5000.1 

Major  System  Acquisition 

DOD  DIR  5000.2 

Defense  Acquisition  Program 
Procedures 

DOD  DIR  5000.29 

Management  of  Computer  Resources  In 
Major  Defense  Systems 

DOD  FAR  Sup  70 

Acquisition  of  Computer  Resources 

D0D~STD-480B 

Configuration  control  -  Engineering 
Changes,  Deviations,  And  Waivers 

DOD-STD-1467 

Software  support  Environment 

D0D-STD-2167A 

Defense  System  Software  Development 

DOD-STD-2168 

Defense  System  Software  Quality 
Program 

DOD  HDBK-287 

Defense  System  Software  Development 
Handbook 

DOD  5000.3-M-3 

Software  Test  and  Evaluation  Master 
Plan  Outline 

MIL-STD-483A 

Configuration  Management  Practices 
For  Systems,  Equipment,  Munitions, 
And  Computer  Programs 

MIL-STD-490A 

Specification  Practices 

MIL-STD-1521B 

Technical  Reviews  and  Audits  for 
Systems,  Ecpilpments,  and  Computer 
Software 
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Pub  Law  89-306 

AFR  800-14 

AFSCP  800-5 

AFSCP  800-14 
AFSCP  800-43 
ESD-TR-88-001 


Brooks  Bill,  Warner-Nunn  Amendment 
and  Implementation  Directives 

Life  Cycle  Management  of  Computer 
Resources  in  Systems 

Software  Development 
Capability/Capaoity  Review 

Software  Quality  Indicators 

Software  Management  Indicators 

Software  Management  Metrics 


SYBttiffi  8fl£tty  .Program 
* 

* 

MIL-STD-882B 
MIL-STO  1574A 

AFSC  OH  1-6 
AFISC  SSH  1-1 
AFR  80-14 
AFR  122-3 

AFR  122-9 

AFR  800-16 
AFSCP  127-1 
AFSCP  800-44 
* 


Air  Force  System  Safety  Handbook 

Program  Manager's  System  Safety 
Guide 

System  Safety  Program  Requirements 

System  Safety  Program  for  Space  and 
Missile  Systems 

Design  Handbook:  System  Safety 

Software  System  Safety  Handbook 

Test  and  Evaluation 

The  Air  Force  Nuclear  Certification 
Program 

Nuclear  Surlty  Design  Certification 
Program  For  Nuclear  Weapon  System 
Software  And  Firmware 

USAF  System  Safety  Program 

System  Safety  Program  Management 

System  Safety  Guidelines 


These  unnumbered  documents  are  published  by  the 
Air  Force  Safety  Agency 
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This  appendix  lists  additional  analysis  techniques  and 
tools  recoininended  by  guidance  documents  or  utilized  by 
various  software  developers  to  assist  In  software  safety 
analysis.  None  of  these  techniques  and  tools,  however,  is 
specifically  designed  to  perform  structured  software  safety 
analysis.  The  list  was  compiled  from  review  of  referenoes 
(8iA'-19,  18:7-14,  19:21,  32:41,  42:B-21)  and  a  survey  of 
aerospace  Industry  personnel. 

•  Cheokllst  of  Common  Software  Errors 

•  Code  and  Module  Walkthroughs 

•  Compare  and  Certification  Tool 

•  Critical  Function  Plows 

•  Cross  Reference  Tools 

•  Data  Flow  Techniques 

•  Design  and  Codo  Inspections 

•  Emulation 

•  llardware/Software  interface  Analysis 

•  Hierarchy  Tool 

•  Integrated  Critical  Path  Analysis 

•  Mathematical  Proofs 

•  Proof  of  Adequacy 

•  Prototyping 

•  Reverse  Engineering  Tools 

•  Simulation 
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Software  Control  Mapping 
Software  Natrloes 

Software  and  Hardware  Allocation  Analysis 

Software  Common  Mode  Analysis 

Source  Code  Analysis 

System  Cross  Check  Matrices 

Test  Coverage  Analysis 

Thread  Analysis 

Top-Down  Keview  of  Code 

Topology  Network  Trees 

Traceability  Matrices 

Requirements  Nodeling/Analysls 
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PERSON  CONTACTED: 


, DATE/TIME; 


PHONE  #: _ 

COMPANV/DOD  AFFILIATION: _ 

POSITION/DUTY  TITLE: _ 

INTERVIEWER : _ 

HOW  MANY  TIMES  HAS  THIS  PERSON  BEEN  CONTACTED: _ 

QUESTIONS: 

1.  Do  you  work  with  th«  DOD  software  davalopmant  prooaas 
aa  daflnad  In  DOD-STO-2167A  or  aoma  oloaa  darlvltlve? 


2.  la  aoftwara  aafaty  or  ayatam  aafety  an  laaua  that  la 
faoad  In  your  organlxatlon? 


3.  la  aoftwara  aafaty  oonaldarad  Internal  or  external  to 
the  overall  aoftwara  development  procaaB? 
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4.  Are  there  any  specific  software  safety  analysis 
techniques  and/or  managenent  processes  related  to 
software  safety  employed  in  your  organization? 
Explain. 


5.  What  successes  and/or  failures  has  your  organization 
had  with  software  safety  analysis  techniques? 


6.  In  your  opinion,  what  are  the  benifits  or  drawbacks  to 
applying  software  safety  analysis  techniques? 


REFERALS ! 

NAME  (i  PHONE: 
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c;  aurviiv  yarii.LO.LDamiB 


1.  Mr  Thomas  D.  Arkwright,  Phd 
System  and  Software  Engineer 
Lockheed  Missiles  and  Space  Company 
Palo  Alto,  CA 

408-756-1861 

2.  Nr  Charles  T.  Fouts 
Staff  Engineer 

Boeing  Defense  and  Space  Group  Helicopters 

Philadelphia,  PA 

215-591-6330 

3.  Mr  Seymour  R.  Friedman 
Associate  Department  Head 
The  MITRE  corp 

Boston,  MA 
617-271-7183 

4.  Mr  Myron  D.  Krueger 
Engineering  Chief 
F-16  Safety  Analysis 

General  Dynamics,  Fort  Worth  Division 
817-763-3306 

5.  Mr  Charles  Lavine 
Software  Safety  Researcher 
The  Aerospace  Corporation 
Los  Angeles,  CA 
310-336-1595 

6.  Captain  Peter  Lemieux 
System  Safety  Manager 
AFSA/SESD 

Norton  AFB,  CA 
DSN  876-4104 

7.  Nr  Roger  Lockwood 
System  Safety  Manager 

Space  and  Missile  Systems  Center 
Los  Angeles  AFB,  CA 
DSN  833-0369 

8.  Mr  Jonathan  F.  Luedeke 
Principal  Research  Engineer 
Battelle 

Columbus,  OH 
614-424-5145 
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9.  Mr  Mltohsll  F.  Lustig 
Dlraotor,  Systam  safety 

Aeronautical  Systems  Center  System  Safety  Office 
Wrlght-Patterson  AFB,  OH 
DSM  785-3694 

10.  Captain  Steven  F.  Mattern 
System  Safety  Manager 

National  Aerospace  Plane  System  Safety  Office 
Wrlght-Patterson  AFB,  OH 
DSN  785-1855 

11.  Mr  Archibald  MoXlnlay  VI 
Software  Safety  Consultant 
MoXlnlay  and  Associates 

St  Louis,  MO 
314-532-2136 

12.  Captain  Xenneth  Miller 
Satellite  Production  Manager 
OL-AP,  GE  Astro-Space  Division 
Princeton,  MJ 
609-490-2466 

13.  Mr  James  W.  Nltohel  Jr. 

Director,  System  Safety 

Air  Force  Development  Test  Center  System  Safety  Office 
Eglin  AFB,  FL 
DSN  872-4157 

14.  Mr  Michael  C.  Moran 

Chairman,  Electronic  Industries  Association 
G-48  System  Safety  Committee 
Lockheed  Corporation 
404-494-2083 

15.  Captain  Oscar  Overton 

Electronic  and  Software  System  Safety  Manager 

USAF/AFOTEC/SEE 

Klrtland  AFB,  NM 

DSN  246-5321 

16.  Mr  Douglas  A.  Peterson 
System  Safety  Engineer 
F-22  system  Program  Office 
Wrlght-Patterson  AFB,  OH 
DSN  785-4502 

17.  Mr  Chris  E.  Phillips 
Safety  Engineer 

General  Dynamics,  Fort  Worth  Division 
817-777-5811 
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18.  Mr  Patrick  Place 

Mextiber  of  the  Technical  Staff 
Software  Engineering  institute 
Carnegie  Mellon  University 
Pittsburgh,  PA  15213 

19.  Mr  William  B.  Rieger 

System  Safety  Engineer/Software  Safety  Engineer 
Boeing  Defense  and  Space  Group 
Seattle,  WA 
206-544-4254 

20.  Mr  Kenneth  Roberts 
Software  Development  Engineer 
OE  Astro-Space  Division 
Princeton,  NJ 
609-951-7334 

21.  Mr  w.  Coy  Sullivan 
Chief,  E-3  OFP  Branch 
Oklahoma  Air  Logistics  center 
Tinker  AFB,  OK 

DSN  336-7044 

22.  Mr  Phillip  C.  Topping 
Software  Engineer 

Lockheed  Missiles  and  Space  Company 

Sunnyvale,  CA 

408-742-2780 

23.  Mr  William  J.  Urschel 

Flight  Systems  Computer  Resources 
B-2  System  Program  Office 
Wrlght-Patterson  AFB,  OK 
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To  maintain  a  software  safety  analysis  focus,  our 
objective  was  to  review  documents  produced  and  delivered 
under  the  300  series  tasks  of  MIL>STD''8B2B  with  Notice  1. 
However,  Notice  1  was  not  introduced  until  July  1987  and 
many  of  the  current  weapon  system  acquisition  programs  were 
con  icted  before  1987.  In  addition,  one  current  contract 
was  let  before  the  1984  publication  of  MIL-STD-882B  and  uses 
M1L-STD-882A.  Only  a  few  programs  have  been  able  to  put  the 
300  series  tasks  on  contract  and  fewer  still  have  received 
the  corresponding  hazard  analysis  reports.  Therefore,  most 
of  the  documents  reviewed,  with  one  exception,  are  results 
of  either  MIL-ST0-882A  or  the  100  and  200  series  tasks  of 
MIL-STD-882B.  The  one  exception  is  the  result  of  Task  301, 
Software  Requirements  Hazard  Analysis. 

The  ASC  system  program  offices  that  participated  In  the 
study  and  the  hazard  analysis  reports  reviewed  are  listed 
below.  Where  applicable  each  document  has  a  parenthetical 
reference  to  Its  associated  MIL-STD-8B2B,  Notice  1  task. 


Cr.I7  ..SYBtftBLErg.grftm-.QXf  icg: 

•  System  safety  Program  Plan,  Rev  E  (Task  101) 

■  System  Hazard  Analysis,  Rev  B  (Task  204) 

•  Avionics  Subsystem  Hazard  Analysis,  Rev  B 
(Task  203) 


•  Fault  Tree  Analysis  of  Erroneous  Flight 
Control  Information  on  the  Heads-Up  Display 

•  Electronic  Flight  Control  Sneak  Circuit 
Analysis  Final  Report,  Rev  A 

These  documents  are  products  of  the  C-17  Full  Scale 

Engineering  Development  (FSED)  phase  which  was  awarded  In 

1981  when  MIL-STD-882A  Was  the  current  system  safety 

requirements  document.  The  C-17  is  currently  In  Its  Low 

Rate  Initial  Production  phase;  therefore,  no  additional 

software  specific  hazard  analyses  have  been  put  on  contract. 

F-16  System  Program  Office; 

•  System  Safety  Hazard  Analysis  (Task  204)  for 
the  F-16/Full  Scale  Engineering  and 
Development  Low  Altitude  Navigation  and 
Targeting  Infrared  for  Night  (LANTIRN) 
Navigation  Pod  System  Wide  Integrity 
Management  (SWIM)  Study 

F-22  System  Program. Of floe: 

«  System  Safety  Program  Plan  (Task  lOl) 

•  Preliminary  Hazard  Analysis  (Task  202) 

•  A  combination  of  a  Subsystem,  System,  and 
Operating  and  Support  Hazard  Analysis  (Tasks 
203,  204,  and  205) 

•  Safety  Assessment  Report  (Task  209) 

These  documents  were  produced  for  the  F-22 
Demonstration/Validation  phase  and  were  used  by  the  program 
office  to  aid  In  the  F-22  aircraft's  proof  of  concept.  The 
F-22  program  has  recently  entered  the  Engineering  and 
Manufacturing  Development  phase  and  MIL-STD-882B  300  series 
tasks  have  been  Imposed  on  the  contractor  developing  team. 
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Software  Requirement  Hazard  Analysis  (Task  301), 
for  the  EF-lllA,  System  Improvement  Program  (SIP) , 
AN/ALQ-99  Encoder/Processor. 
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1. 


Mr  Eldon  Bertran 
Modern  Technologies  Corporation 
System  Safety  Management  Contractor 
F-22  System  Program  Office 
Wright-Patterson  AFB,  OH  45433 
DSN  785-4502 

2.  Major  Robert  J.  Congelli 
Director,  System  Safety 
G-17  System  Program  Office 
Wright-Patterson  AFB,  OH  45433 
DSN  785-6719 

3.  Lt  Col  Harry  V.  Dutohyshyn 
Chief,  F-16  System  Safety  Division 
F-16  System  Program  Office 
Wright-Patterson  AFB,  OH  45433 
DSN  785-1938 

4.  Mr  Jon  D.  Kooara 
Director,  System  Safety 

Eleotronio  Combat  &  Reconnaissance  Sys  Program  Office 
Wright-Patterson  APB,  OH  45433 
DSN  785-9249 

5.  Captain  Steven  F.  Nattern 
Chief,  System  Safety  Management 
HASP  system  Program  Office 
Wright-Patterson  AFB,  OH  45433 
DSN  785-1855 

6.  Mr  Douglas  A.  Peterson 
System  Safety  Engineer 
F-22  System  Program  Office 
Wright-Patterson  AFB,  OH  45433 
DSN  785-4502 
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For  more  information  on  the  ASC  System  Safety  Training 


Course  contact t 


Mr  Mitchell  F.  Lustig 
Director,  System  Safety 
ASC/EMSS 

Wright-PAtterson  AFB,  OH  45433 
DSN  785-3694 


Air  Force  Sponsored  Courses 

For  complete  course  descriptions,  prerequisites,  course 
lengths,  and  locations  of  Air  Force  sponsored  courses  refer 

to  the  latest  version  of  AFR  50-5,  Tgfllninqs _ USAF  Formal 

aflhoQii. 

For  information  on  SAS  001,  002,  and  006  courses 
contact ! 

6575th  School  Squadron  (AFMC) 

Brooks  AFB,  TX  78235-5000 
DSN  240-2102 


NttYY.,.SPflnigrid  Cfl-ursia 

For  more  Information  on  the  system  safety  courses 

conducted  by  the  Naval  Safety  School  contact: 

Naval  Safety  Schcol 
Naval  Air  Station 
Norfolk,  VA  23511-5410 
DSN  565-8778 
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For  inor«  information  on  th«  System  Safety  Course  and 

the  Software  Safety  Course  contact! 

University  of  Southern  California 
Institute  of  Safety  and  Systems  Management 
927  West  3Sth  Place 
Room  102 

Los  Angeles,  CA  900B9-0021 
Phone!  213-740-3995 

For  more  information  on  the  Software  Safety  Workshop 
contact! 


University  of  Callfornla-Los  Angeles 

Extension  Building 

Short  Course  Program  Office 

10995  Le  Conte  Avenue 

Suite  542 

Los  Angeles,  CA  90024-2883 
Phonet  310-825-3344 
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